In the past decades there have been myriads of applications in substations, ranging from SCADA, control, automation, maintenance, protection and others, which have relied upon protocols using serial connectivity over serial port RS-232 or RS-485 interface / links. These were point-to-point or point-to-multipoint configurations, with speeds of tens of kbit/s at best. The technology was simple, easy to debug and troubleshoot and reasonably reliable.
Substation applications have been run and managed in this way for some decades. As long as there were not additional applications or increased requirements for existing applications, there was no need to embrace a technology change.
Now, new substation automation techniques and approaches driven by new standards such as IEC 61850 have originated that technology change. New substations are built using IP-native protocols, and as a result of this many utilities face a common problem at the time of maintaining the existing infrastructure:
Is it better to try to refurbish old substations from scratch and replace all functions and devices with modern ones, or to try to integrate legacy infrastructure and applications into the new all-IP substation system? Are the operational gains worth the extra investment?
How to implement a centralized and automated control to all legacy devices like RTUs, protection relays and control IEDs using the IP / Ethernet network?
Shall the existing legacy communication be moved to IP infrastructure at all? Or shall they be left running on the existing infrastructure, until their operational lifetime has expired and „proper‟ IP solution can be implemented? Leaving the legacy data on existing infrastructure avoids the need for any conversion/integration on the level of data communication. But of course the question of integration then comes at the application level. Example: A SCADA system (Frontend) must support the handling of e.g. IP based data (IEC 60870-5-104) and existing serial data (IEC 60870-5-101) These questions must be analyzed taking into account both operational and economic factors. In general, different paths can be observed depending on the application and sector, as seen in the survey results in chapter
0
. However, many times the migration process is necessary, and some legacy systems must be integrated into the IP architecture. In these scenarios there are some common issues that have to be addressed in order to successfully integrate the infrastructure.3.4.1 Considerations when migrating serial port based systems to IP/Ethernet interface
In addition to business and operational requirements, once a migration of a legacy application to IP has been approved, there are some technical issues that have to be carefully analyzed in order to ensure that the legacy application is going to run correctly over the new IP infrastructure. This is due to the differences in nature between packet-switching oriented networks (e.g. IP) and circuit switching or serial port based links.
Applications and protocols running over serial ports share a set of common characteristics due to the nature of the physical
Negligible latency in comparison with the bit speed used (<100kbps), as is the case with a copper wire a few meters in length or optical fiber up to hundreds of meters in length. Then these applications may suffer when they are carried over an IP network with latencies in the order of several milliseconds or worse. If they have to traverse an IP router, the application may not work at all due to increased and random latency. So latency control is critical
Strict timing requirements. The IP network may have an irregular distribution of latency (variance of the latency) depending on network congestion and actual traffic in each device. This may harm some applications that need strict inter-bit and inter-frame timings, initially designed for a serial port medium where this behavior does not happen or may be assimilated to a broken or damaged link
Link Layer Control (LLC) mechanisms, which take care of error correction or detection and sequence control. Thus, some applications suffer heavily when encapsulated over the TCP protocol, which guarantees sequenced, error-free information delivery, usually at the cost of high latency and retries.
UDP protocol is usually a better choice for the transmission of encapsulated legacy serial protocols, as generally it is better for serial-port based applications to receive a wrong frame in time, than to receive the correct frame too late, when link layer timers have expired
Serial protocols often have a well-defined frame format, which must be analyzed by the devices making the serial-to-IP conversion. Whenever possible protocol frame fragmentation must be avoided, and it is often helpful - though more bandwith consuming - to encapsulate just one frame per transmitted datagram. This procedure ensures smooth protocol timing between frames. As a result of this, frame start and stop delimiters and frame length fields should be decoded by the IP encapsulator As a rule of thumb legacy applications that match the following 3 criteria can be run smoothly when encapsulated over UDP connections in most legacy serial port oriented substation applications:
There is a frame format correctly defined by characters or binary sequences at the beginning and end of the frame, avoiding ambiguities (predefined escape sequences, no possible ambiguities)
There are predetermined maximum and minimum frame lengths, and where possible not too long frames are used, as this impacts negatively on latency. As an example, satisfactory results have been obtained for many tele-control protocols with a maximum frame value of 256 bytes
There is an error detection mechanism in the protocol frame itself (CRC or similar) which allows for validation of the integrity of the protocol frame at both ends. This allows for filtering of error-containing frames before delivering them over the serial port
4 Defining the Network Architecture
From the functional point of view there are two types of communications services used by electric power companies:
Corporate or enterprise services Operational services
The corporate services are used for the applications related to the Utility enterprise, market related activities and in some cases U-Telco activities (utility company being a telecommunications operator). The corporate services are not described in the document as the focus is only on the operational services.
The purpose of the operational services is to use telecommunications network and technology in order to maintain reliable, secure and efficient core operations of electric power utility. The list of typical applications realized with operational communications services is provided below:
Protection and Control SCADA
EMS/DMS
Wide Area Monitoring Systems Operational Telephony Systems Substation Data Retrieval
Remote Access to Devices and Asset Management Condition and Quality Monitoring
Mobile Workforce Management Operational Cyber Security
Operational Physical Security and Video Surveillance
Some of the applications listed above like protection and control (teleprotection or line differential protection) are well established functions practiced for many years that are critical to maintain safe and reliable
operation of the electrical grid. However some of the applications are emerging functions that are becoming more and more important for improving utilities‟ efficiency and ensuring more intelligent decision making.
These new applications include condition monitoring, mobile workforce management, etc.
Electric utilities have the choice to integrate operational and non-operational services in a combined communications infrastructure or run these services in completely separate networks. There are three possibilities how the different communications services can be realized :
Operational and corporate services share the same network and there is no separation
Operational and corporate services share the same network, that has logical separation with SLA (Service Level Agreement) management
Operational and corporate services are implemented on physically separate networks
In large utilities the operational network is a combination of a Wide Area Network (WAN) and multiple Local Area Network Networks (LAN). The WAN network forms a communications backbone connecting together all major substations with the national and regional control centers. The LAN networks typically are contained within subststations and are interconnecting devices inside the substation perimeter and/or a number of remote devices located in its geographical proximity such as various sensors, controllers and smart meters.
Communications Backbone (Core Network) Regional Control
Centers National Control
Centers
Substation LAN Ethernet
Switch
Router
End Devices Substations