Here - Hydro
February 6, 2018 | Author: Anonymous | Category: N/A
Short Description
Download Here - Hydro...
Description
Cisco and Hydro Québec Use cases proposal for IEC 61850-90-12
Faramarz Maghsoodlou, Ph. D. IoT Connected Industries & Energy Practice Advanced Services Cisco Systems, Inc.
Jean Raymond, ing., Ph. D., M. Sc. A. Evolution du Réseau IP, STAR Direction Ingénierie et Solutions de Télécommunications Direction Principale Télécommunications - Groupe Technologie Hydro-Québec
August 26, 2014
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
1
Table of Contents 1.
Preface....................................................................................................................................... 5
2.
Introduction ............................................................................................................................... 6
3.
Evolution of the Utility Telecom Networks.................................................................................. 9 3.1 3.2 3.3 3.4 3.5 3.6 3.7
4.
Transmitting Data over Multiple Media .............................................................................................9 Changing and Growing with the Industry ......................................................................................... 10 Connecting Large Number of Devices .............................................................................................. 10 Maintaining Reliability .................................................................................................................... 10 Connecting Multiple Types of Systems ............................................................................................. 10 Ensuring Security ............................................................................................................................ 11 Providing Smooth Migration............................................................................................................ 11
Use Cases ................................................................................................................................. 13 4.1 Transmission Use Cases ................................................................................................................... 16 4.1.1 Tele-Protection ................................................................................................................................... 16 4.1.2 Inter-Trip Protection Scheme ............................................................................................................. 19 4.1.3 Current Differential Protection Scheme ............................................................................................. 23 4.1.4 Distance Protection Scheme ............................................................................................................... 25 4.1.5 Inter-Substation Protection Signaling ................................................................................................. 27 4.1.6 Intra-Substation Process Bus Communication.................................................................................... 30 4.1.7 Control Center Monitoring of Station Bus .......................................................................................... 33 4.1.8 Communication System Failure & Degradation Monitoring .............................................................. 35 4.1.9 High Voltage Substation SCADA RTU .................................................................................................. 38 4.1.10 Medium Voltage Substation SCADA RTU.......................................................................................... 41 4.1.11 Condition-Based Monitoring ............................................................................................................ 44 4.1.12 Transformer Dissolved Gas Analysis ................................................................................................. 47 4.1.13 Gas-Insulated Switchgear Health Monitoring .................................................................................. 49 4.1.14 Dynamic Asset Rating ....................................................................................................................... 51 4.1.15 Wide Area Monitoring & Control Systems ....................................................................................... 54 4.1.16 Substation Environmental Monitoring ............................................................................................. 58 4.1.17 Power System State Estimation........................................................................................................ 61 4.1.18 Inter Control Center Communication (ICCP) .................................................................................... 64 4.1.19 RTU File Transfer............................................................................................................................... 68 4.1.20 Weather Monitoring Stations ........................................................................................................... 70 4.2 Distribution .................................................................................................................................... 72 4.2.1 Power Quality Monitoring System – PQMS ........................................................................................ 72 4.2.2 Sensors in the Distribution Grid .......................................................................................................... 75
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
2
4.2.3 Pole-Top Voltage Regulators & Capacitor Bank Monitoring .............................................................. 76 4.2.4 Remote Control of Overhead or Underground Switches ................................................................... 79 4.2.5 Distribution Feeder Voltage Regulation ............................................................................................. 81 4.2.6 Distribution Volt / VAR Optimization.................................................................................................. 84 4.2.7 Substation Capacitor Monitoring & Control ....................................................................................... 87 4.2.8 Underground Cable Distributed Temperature Monitoring ................................................................ 88 4.2.9 Online Transformer Condition Monitoring ......................................................................................... 92 4.2.10 Switchgear & Transformer Partial Discharge Monitoring ................................................................ 94 4.2.11 Fault Location Isolation & Service Restoration (FLISR) ..................................................................... 96 4.2.12 Voltage Regulation............................................................................................................................ 99 4.2.13 Advanced Metering Infrastructure (AMI) ....................................................................................... 103 4.2.14 Integrated Grid-Scale Energy Storage ............................................................................................ 106 4.3 Energy Supply Use Cases ............................................................................................................... 109 4.3.1 Frequency Control / Automatic Generation Control (AGC) .............................................................. 109 4.3.2 Hydroelectric Power Plants – General Telemetry............................................................................. 112 4.3.3 Hydro Plant Dam Leakage Monitoring .............................................................................................. 115 4.3.4 Hydro Plant Gate Position Indicator ................................................................................................. 117 4.3.5 Hydro Plant Water Flow Control ....................................................................................................... 119 4.3.6 Hydro Plant Water Level Indicator.................................................................................................... 121 4.3.7 Hydro Plant Dam Over-Topping Protection ...................................................................................... 123 4.3.8 Hydro Plant Turbine Vibration Monitoring ....................................................................................... 125 4.3.9 Hydro Plant Dam Deformation Monitoring ...................................................................................... 126 4.3.10 Wind Farm Operation ..................................................................................................................... 127 4.3.11 Wind Farm Maintenance ................................................................................................................ 130 4.4 Extreme Contingencies .................................................................................................................. 133 4.4.1 H-Q Transmission System Characteristics......................................................................................... 133 4.4.2 Extreme Contingencies Criteria ........................................................................................................ 134 4.4.3 Special Protection Schemes (SPS) ..................................................................................................... 136 4.4.4 Telecom Requirements for Extreme Contingencies ......................................................................... 138 4.4.5 NPCC requirements for SPS Communication Systems...................................................................... 139 4.5 Demand Side ................................................................................................................................ 141 4.5.1 Hour-Ahead Load Optimization – Demand Response ...................................................................... 141 4.5.2 Electric Vehicle Charging................................................................................................................... 143 4.5.3 Automated Demand Response with Water Heaters ........................................................................ 145 4.5.4 Customer Premise Network Integration ........................................................................................... 150 4.6 Mobility & Collaboration ............................................................................................................... 151 4.6.1 Field Workforce Voice over IP (VoIP) ................................................................................................ 151 4.6.2 Workforce Video ............................................................................................................................... 153 4.6.3 Radio over IP (RoIP) .......................................................................................................................... 155 4.6.4 Substation Worker Access to Corporate Applications ...................................................................... 158 4.7 Physical Safety & Security ............................................................................................................. 161 4.7.1 Electronic Access Control .................................................................................................................. 161 4.7.2 Video Monitoring & Surveillance ...................................................................................................... 164 4.7.3 Remote Fire Alarms Monitoring ....................................................................................................... 167 Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
3
4.8 Other Use Cases ............................................................................................................................ 169 4.8.1 Generic Network Management Use Case ......................................................................................... 169 4.8.2 Precision Time Protocol (PTP) ........................................................................................................... 172
5.
Network Requirements Heat Map & Dependency ................................................................... 173 5.1 5.2
6.
Network Requirements Heat Map – By Use Case ........................................................................... 174 Network Dependency Matrix – By Use Case ................................................................................... 176
Communication Trends & Best Practices ................................................................................. 178 6.1 General communication Requirements .......................................................................................... 178 6.2 Migration to Packet-Switched Network ......................................................................................... 182 6.3 MPLS technology .......................................................................................................................... 183 6.3.1 Network Virtualization ...................................................................................................................... 183 6.3.2 Support for Existing Networks & Interoperability ............................................................................ 184 6.3.3 Security in MPLS................................................................................................................................ 184 6.3.4 Utility-Grade Performance................................................................................................................ 184 6.3.5 IP/MPLS and MPLS-TP for the WAN ................................................................................................. 185 6.3.6 The Cost Efficiencies of MPLS ........................................................................................................... 185 6.4 IP Address Planning & Management .............................................................................................. 186 6.4.1 Overview ........................................................................................................................................... 186 6.4.2 IPv6 Considerations .......................................................................................................................... 187 6.4.3 Critical Issues in IP Address Assignment ........................................................................................... 187
7.
Security Trends & Best Practices ............................................................................................. 188 7.1 7.2 7.3 7.4
Current Practices & Their Limitations ............................................................................................. 188 Security Trends in Utility Networks................................................................................................ 189 Regulatory Compliance (NERC CIP) ................................................................................................ 190 General Security Requirements ..................................................................................................... 193
8.
Bibliography ........................................................................................................................... 197
9.
Glossary ................................................................................................................................. 198
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
4
1. Preface The use cases that are described in this document represent an abridged version of the content that was developed by Faramarz Maghsoodlou of Cisco Systems’ Advanced Services, in collaboration with Jean Raymond of Hydro Québec (H-Q), as part of H-Q’s effort to prepare for, plan, and deploy the next generation of its IP/MPLS telecommunications network. Cisco and Hydro Québec have jointly decided to make this document available, in a limited distribution, to the IEC 61850-90-12 committee members to assist in their collective work and deliberations for developing the relevant IEC standards. The content of this document, in its entirety, including methods, diagrams, figures, and descriptions, is the intellectual property of Cisco Systems, Inc. and it is not intended for public distribution. Its use and distribution is limited to the collective work of the IEC committee members for the purpose of standards development. References to this document should be cited as: Faramarz Maghsoodlou and Jean Raymond, “Cisco and Hydro-Québec Use cases proposal for IEC 61850-90-12, August 2014.”
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
5
2. Introduction The business and technology trends that are sweeping the utility industry will drastically transform the utility business from the way it has been for many decades. At the core of many of these changes is a drive to modernize the electrical grid with an integrated communications infrastructure. At Hydro-Québec, the Direction Principale des Télécommunications group (DPT) launched a private MPLS network design and implementation project a few years ago to provide H-Q with advanced network services. This project and this document are a part of that effort. This document consists of 9 chapters that cover the following topics: • • • • • • • • •
Introduction Evolution of the Utility Telecom Networks Use Cases that represent eight utility operational domains Network Requirements Heat Map & Dependencies matrix General Communication Trends & Best Practices General Security Trends & Best Practices Smart Grid Standards Bibliography Glossary
Given the range and diversity of the requirements that should be addressed by the next generation telecommunications infrastructure, the approach that we adopted in this project is to document the telecommunication requirements based on a wide range of operational use cases that reflect the current and future needs of the generation, transmission, and distribution business. These future operational needs are themselves a reflection of the business and technology trends that are shaping the industry. We therefore started by reviewing the top 15 industry trends and explored their relevance to Hydro Québec. Top Business & Technology Trends IT-OT Convergence Field Area Network Advanced Metering Infrastructure Mobility & Mobile Technologies Business Intelligence & Big Data Analytics Social Computing Electric Vehicles XaaS and Cloud Computing
Figure 1.
Energy Storage Distributed Intelligence Transactive Energy Aging Workforce Aging Assets Infrastructure Security Renewable Energy
Top Energy Industry Busienss & Technology Trends
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
6
For each trend we describe the business and technology drivers and point out challenges that utilities face in responding to these changes. We offer recommendations on how Hydro Québec could address these challenges based on their relevance and level of impact on H-Q’s operations and business model. The following figure shows the impact of these trends on H-Q on a variable scale of 0 to 5, with 0 indicating no impact and 5 indicating a high level of impact.
IT-OT Convergence 5 Renewable Resources Field Area Network 4
Advanced Metering Infrastructure (AMI)
Infrastructure Security 3 2
Aging Assets
Mobility & Mobile Technologies
1 0
Business Intelligence (BI) & Big Data Analytics
Aging Workforce
Transactive Energy
Social Computing
Distributed Intelligence Energy Storage
Figure 2.
Electric Vehicles XaaS and Cloud Computing
Top Industry Trends & Their Relevance to Hydro Québec
To meet this diverse set of requirements, both today and in the future, it is imperative that the next generation utility telecom network will be based on open-standards-based IP architecture. The future end-to-end IP/MPLS architecture will enable Hydro Québec to: • • • • • • •
Facilitate interoperability across disparate networks and devices Support data transmission over diverse media Connect large number of devices Maintain reliability Connect multiple types of systems Ensures security, and Change and grow with the industry.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
7
Focusing on the operational requirements, our starting point is set of well-defined use cases that are grouped into the following operational domains:
Transmission Distribution Energy Supply Extreme Contingencies
Figure 3.
Operational Domains Demand Side Mobility & Collaboration Physical Safety & Security Other
Operational Domains for Use Case Classification
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
8
3. Evolution of the Utility Telecom Networks The business and technology trends that are sweeping the utility industry will drastically transform the utility business from the way it has been for many decades. At the core of many of these changes is a drive to modernize the electrical grid with an integrated communications infrastructure. However, interoperability, concerns, legacy networks, disparate tools, and stringent security requirements all add complexity to grid transformation. Given the range and diversity of the requirement that should be addressed by the next generation telecommunications infrastructure utilities need to adopt a holistic architectural approach to integrate the electrical grid with digital communication across the entire power delivery chain. Many utilities still rely on complex environments formed of multiple application-specific, proprietary networks. Information is siloed between operational areas. This prevents utility operations from realizing the operational efficiency benefits, visibility, and functional integration of operational information across grid applications and data networks. The key to modernizing grid communications is to provide a common, multi-service network infrastructure for the entire utility organization. Such a network serves as the platform for current capabilities while enabling future expansion of the network to accommodate new applications and services. To meet this diverse set of requirements, both today and in the future, the next generation utility telecom network will be based on open-standards-based IP architecture. An end-to-end IP architecture takes advantage of nearly three decades of IP technology development, facilitating interoperability across disparate networks and devices, as it has been already demonstrated in many mission-critical and highly secure networks. It is imperative that utilities participate in standards development bodies to influence the development of future solutions and to benefit from shared experiences of other utilities and vendors. In the following sections we indicate how IP can meet every challenge that utilities will face in their grid modernization journey.
3.1 Transmitting Data over Multiple Media Utility operational data Smart grid data must be able to travel rapidly and reliably over a variety of different network media, from copper cables to fiber infrastructure to wireless networks. IP can run over any link layer network, including Ethernet, wireless radio networks, and serial lines, providing a common and flexible way to use and manage a network composed of disparate parts.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
9
3.2 Changing and Growing with the Industry The electric power system will evolve as technological advances yield new hardware, applications, and devices. At the same time, the utility must incorporate such advances into the network with minimal cost and difficulty. One of the principal benefits of IP is its ability to add a capability such as a new application without having to change IP itself. A good analogy is a highway and cars: car designs change constantly in response to emerging consumer demands, but nonetheless can still use the same roads and traffic management. That is why IP can run applications it was not originally designed to support, such as secure Internet commerce, voice, collaboration, and Web 2.0 applications. And just as highways are designed to support traffic for the next 100 years, IP will be able to support new applications as they are developed for decades to come.
3.3 Connecting Large Number of Devices 1 The telecom infrastructure must enable communication and correlation of data from potentially thousands of substations and many millions of grid and consumer devices. One of the main challenges with connecting large numbers of devices is providing a unique identifier, or address, for each device. Unlike the many architectures that went before it, IPv6 offers straightforward addressing and routing for a huge network such as the future utility telecom network.
3.4 Maintaining Reliability High network availability is absolutely critical. Network outages are costly and debilitating – and unfortunately all too frequent these days. Ensuring uninterrupted electrical service to ratepayers is a prime challenge for any utility. Therefore, ensuring that the utility telecom network is reliable, so that it in turn can ensure uninterrupted electrical service to ratepayers, is crucial. IP already has more tools and applications to help manage the network and maintain reliability than any other communication protocol.
3.5 Connecting Multiple Types of Systems The utility communication network must connect and exchange data freely with many different types of hardware, ranging from smart sensors in home appliances to home energy meters to transformers and beyond.
1
A Standardized and Flexible IPv6 Architecture for Field Area Networks
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
10
IP is device independent. This means that it can identify any type of system to which data is addressed and deliver it to its destination. IP can also identify the system from which the data came, so it enables the receiving device to respond back to the sending device to let it know the data has arrived.
3.6 Ensuring Security The unfortunate reality is that because of the critical nature of the technology and the services it provides, the grid becomes a prime target for acts of terrorism and cyber attacks. The transformation of traditional energy networks to smart grids requires an intrinsic security strategy and specific security mechanisms to safeguard this critical infrastructure. IP is as secure as you want to make it. Although IP was designed to be open and flexible, over the years more and more tools have been built to provide security in the communications that travel over an IP network. In fact, of all communications protocols, IP has the most tools for securing and managing the transport of data. Therefore, while all the communications systems in the network will be able to utilize IP as a communications pipeline, IP has state-of-the-art tools to ensure the information travels as privately as needed, sending the information to the right destination while ensuring that it is not intercepted or accessed by unauthorized users. IP is able to provide security on both public and private networks, and today many industries transmit their communications over both these types of networks. For example, some parts of financial networks are public as well as very secure, such as the retail banking section, while at the same time, many other areas of financial networks are completely private. All of these networks utilize IP as their foundation. Many industries with exacting security standards have embraced IP, despite initial reservations. For example, governments, militaries, service providers for both voice and cable services, telecommunications providers, and mainframe computer utilities were at first concerned about using IP for their operations, fearing security risks. Now all of these industries use IP as their communications foundation. IP has adapted to meet the stringent requirements of their networks, especially in the area of security.
3.7 Providing Smooth Migration Utilities must be able to migrate from their current disjointed data communications networks to converged networks in a phased approach with minimal service effect. IP provides a way to migrate in phases from multiple monitoring and control networks to a single converged network without disrupting service. This enables utilities to receive all the benefits of IP without having to undergo a massive “forklift” implementation. The steps to the convergence are: •
Encapsulation – Legacy non-routable data communications protocols are encapsulated in an IP “wrapper,” which can then be routed over an IP network. One way to accomplish this is bisync serial tunneling, or BSTUN, a protocol originally designed to facilitate migration from mainframe System Network Architecture (SNA) networks to IP. Not only is this method an effective first step in IP
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
11
migration, it also emphasizes IP’s proven track record of flexibility and reliability. However, encapsulation does not offer the kind of end-to-end manageability and high performance afforded by native IP, but it is a relatively straightforward and easy to implement first step. •
Gateways – Protocol translation devices, called gateways, are installed between legacy networks and the IP network. The gateway maps the legacy protocol functions to IP functions. Protocol translation is not a format conversion operation. Rather, it is similar to translating between two human languages— not every word or phrase in one language has an equivalent in the other. Some words and phrases cannot be translated at all, so the translator simply does its best to supply as close a translation as possible. Therefore, like encapsulation, gateways are a useful but temporary migration step.
•
Native IP – The ultimate goal of migration is a native IP network. A native IP network provides the endto-end robust security and outstanding manageability discussed above, along with quality of service, redundancy, scalability, and adaptability. A native IP network also delivers benefits of lower operational expense due to easier implementation and streamlined management.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
12
4. Use Cases The list of use cases is divided into two categories of in-scope and out-of-scope based on their relevance to Hydro Québec’s business and operational requirements. The following tables summarize the in-scope use cases in the respective business categories. For each use case we show three timelines: 1. The rollout timeline of the functionality at Hydro Québec with any communication technology 2. The anticipated rollout timeline of the functionality at Hydro Québec with IP / MPLS technology2 3. The anticipated rollout timeline of the functionality at industry-leading utilities
Rollout Timeline Hydro Québec With Any Technology With IP / MPLS
Transmission Use Cases Tele-Protection Inter-Trip Protection Scheme Current Differential Protection Scheme Distance Protection Scheme Inter-Substation Protection Signaling Process Bus Communication Control Center Monitoring of Station Bus Communication System Failure & Degradation Monitoring High Voltage Substation SCADA RTU Medium Voltage Substation SCADA RTU Condition-Based Monitoring Transformer Dissolved Gas Analysis Gas-Insulated Switchgear Health Monitoring Dynamic Asset Rating Wide Area Monitoring & Control Systems Substation Environmental Monitoring Power System State Estimation Inter Control Center Communication (ICCP) RTU File Transfer Weather Monitoring Stations
Figure 4.
2
Currently Deployed Currently Deployed Currently Deployed Currently Deployed 0 - 5 Years 5 - 10 Years 5 - 10 Years Currently Deployed Currently Deployed Currently Deployed 0 - 5 Years 0 - 5 Years 0 - 5 Years Currently Deployed 0 - 5 Years Currently Deployed Currently Deployed Currently Deployed Currently Deployed 0 - 5 Years
5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 0 - 5 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years
Industry Leaders With IP / MPLS 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years Currently Deployed Currently Deployed Currently Deployed 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years Currently Deployed Currently Deployed 0 - 5 Years Currently Deployed 0 - 5 Years 0 - 5 Years
Transmission Use Cases Rollout Timeline
This rollout timeline with IP/MPLS can vary based on the orientations and the priorities of the utility.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
13
Rollout Timeline Hydro Québec With Any Technology With IP / MPLS
Distribution Use Cases Power Quality Monitoring System – PQMS Sensors in the Distribution Grid Pole-Top Voltage Regulators & Capacitor Bank Monitoring Remote Control of Overhead or Underground Switches Distribution Feeder Voltage Regulation Distribution Volt / VAR Optimization Substation Capacitor Monitoring & Control Underground Cable Distributed Temperature Monitoring Online Transformer Condition Monitoring Switchgear & Transformer Partial Discharge Monitoring Fault Location Isolation & Service Restoration (FLISR) Voltage Regulation Advanced Metering Infrastructure (AMI) Integrated Grid-Scale Energy Storage
Figure 5.
Rollout Timeline Hydro Québec With Any Technology With IP / MPLS
Frequency Control / Automatic Generation Control (AGC) Hydroelectric Power Plants – General Telemetry Hydro Plant Dam Leakage Supervision Hydro Plant Gate Position Indicator Hydro Plant Water Flow Control Hydro Plant Water Level Indicator Hydro Plant Dam Over-Topping Protection Hydro Plant Turbine Vibration Monitoring Hydro Plant Dam Deformation Monitoring Wind Farm Operation Wind Farm Maintenance
Figure 6.
Extreme Contingencies Use Cases
Currently Deployed Currently Deployed Currently Deployed Currently Deployed Currently Deployed Currently Deployed Currently Deployed Currently Deployed Currently Deployed Currently Deployed 5 - 10 Years
5 - 10 Years Currently Deployed 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years Currently Deployed 0 - 5 Years
Industry Leaders With IP / MPLS
5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years
5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years
Distribution Use Cases Rollout Timeline
Rollout Timeline Hydro Québec With Any Technology With IP / MPLS Currently Deployed
Figure 7.
5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years
Distribution Use Cases Rollout Timeline
Energy Supply Use Cases
Special Protection Schemes
0 - 5 Years 5 - 10 Years 5 - 10 Years 0 - 5 Years Currently Deployed Currently Deployed 5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years 0 - 5 Years Currently Deployed Currently Deployed 0 - 5 Years
Industry Leaders With IP / MPLS
Industry Leaders With IP / MPLS
5 - 10 Years
5 - 10 Years
Extreme Contingencies Use Cases Rollout Timeline
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
14
Rollout Timeline Hydro Québec With Any Technology With IP / MPLS
Demand Side Use Cases Hour-Ahead Load Optimization – Demand Response Electric Vehicle Charging Demand Response with Water Heaters Using OpenADR2 Customer Premise Network Integration
Figure 8.
Field Workforce Voice over IP (VoIP) Workforce Video Radio over IP (RoIP) Substation Worker Access to Corporate Applications
Figure 10.
5 - 10 Years 5 - 10 Years 0 – 5 Years 5 - 10 Years
Rollout Timeline Hydro Québec With Any Technology With IP / MPLS 0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years
0 - 5 Years 0 - 5 Years 0 - 5 Years 0 - 5 Years
Industry Leaders With IP / MPLS Currently Deployed 0 - 5 Years Currently Deployed Currently Deployed
Mobility & Collaboration Use Cases Rollout Timeline
Physical Safety & Security Use Cases Electronic Access Control Video Monitoring & Surveillance Remote Fire Alarms Monitoring
5 - 10 Years 5 - 10 Years 5 - 10 Years 5 - 10 Years
Demand Side Use Cases Rollout Timeline
Mobility & Collaboration Use Cases
Figure 9.
5 - 10 Years 0 - 5 Years 0 - 5 Years 5 - 10 Years
Industry Leaders With IP / MPLS
Rollout Timeline Hydro Québec With Any Technology With IP / MPLS Currently Deployed Currently Deployed Currently Deployed
0 - 5 Years 0 - 5 Years 0 - 5 Years
Industry Leaders With IP / MPLS Currently Deployed Currently Deployed Currently Deployed
Physical Safety & Security Use Cases Rollout Timeline
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
15
4.1 Transmission Use Cases This section includes use cases related to power transmission including protection, substation automation and centralized grid monitoring and control through EMS/SCADA.
4.1.1 Tele-Protection The key criteria for measuring Teleprotection performance are command transmission time, dependability and security. These criteria are defined by the IEC standard 60834 as follows: •
•
•
Transmission time (Speed): The time between the moment where state changes at the transmitter input and the moment of the corresponding change at the receiver output, including propagation time. Overall operating time for a Teleprotection system includes the time for initiating the command at the transmitting end, the propagation time over the communications link and the selection and decision time at the receiving end, including any additional delay due to a noisy environment. Dependability: The ability to issue and receive valid commands in the presence of interference and/or noise, by minimizing the probability of missing command (PMC). Dependability targets are typically set for a specific bit error rate (BER) level. Security: The ability to prevent false tripping due to a noisy environment, by minimizing the probability of unwanted commands (PUC). Security targets are also set for a specific bit error rate (BER) level.
Additional key elements that may impact Teleprotection performance include bandwidth rate of the Teleprotection system and its resiliency or failure recovery capacity. Transmission time, bandwidth utilization and resiliency are directly linked to the communications equipment and the connections that are used to transfer the commands between relays.
4.1.1.1 Latency Budget Considerations Delay requirements for utility networks may vary depending upon a number of parameters, such as the specific protection equipment used. Most power line equipment can tolerate short circuits or faults for up to approximately five power cycles before sustaining irreversible damage or affecting other segments in the network. This translates to total fault clearance time of 100ms. As a safety precaution, however, actual operation time of protection systems is limited to 70- 80 percent of this period, including fault recognition time, command transmission time and line breaker switching time. Some system components, such as large electromechanical switches, require particularly long time to operate and take up the majority of the total clearance time, leaving only a 10ms window for the communications part of the protection scheme, independent of the distance to travel. Given the sensitivity of the issue, new networks impose requirements that are even more stringent: IEC standard 61850 limits the transfer time for protection messages to 1⁄4 - 1⁄2 cycle or 4 - 8ms (for 60Hz lines) for the most critical messages.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
16
The following diagram shows the latency budget for a fault clearing time of a protection system, divided by the different actors involved.
Figure 11.
Tele-Protection Clearance Time
As it can be seen, most of the delay is caused by the electromechanical actions on the circuit breaker itself, leaving a very short delay budget for the communications portion.
4.1.1.2 Asymmetric Delay In addition to minimal transmission delay, a differential protection communication channel must be synchronous, i.e., experiencing symmetrical channel delay in transmit and receive paths. This requires special attention in jitter-prone packet networks. While optimally Teleprotection systems should support zero asymmetric delay, typical relays can tolerate discrepancies of up to 250µs. H-Q’s existing relays can tolerate up to 750µs. The main tools available for lowering delay variation below this threshold are: •
•
A jitter “buffer” at the multiplexers on each end of the line can be used to offset delay variation by queuing sent and received packets. The length of the queues must balance the need to regulate the rate of transmission with the need to limit overall delay, as larger buffers result in increased latency. This is the old TDM traditional way to fulfill this requirement. Traffic management tools ensure that the Teleprotection signals receive the highest transmission priority and minimize the number of jitter addition during the path. This is one way to meet the requirement in IP networks.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
17
•
Standard Packet-Based synchronization technologies, such as 1588-2008 Precision Time Protocol (PTP) and Synchronous Ethernet (Sync-E), can help maintain stable networks by keeping a highly accurate clock source on the different network devices involved.
4.1.1.3 Legacy Teleprotection vs. Next generation Teleprotection Requirements The following table captures the main differences between legacy teleprotection and next generation teleprotection requirements. The next generation teleprotection will be based on the IEC 61850 standard.
Network Requirements Client Interfaces One Way Delay Delay Symmetry Required Jitter Layer 2 or Layer 3 VPN Topology Multicast Availability / Reliability Precise Timing Required Link / Node Failure Delay Performance Management
Legacy Teleprotection C37.94, X.21, E&M, G.703, T1/E1, V.35, RS422 4 to 10ms Yes 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Date August 26, 2014
X X Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
23
Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
Communication Events & Attributes Event Sample Value of Current Mechanism Protocol Bandwidth EventProprietary 64 kbps triggered Redundancy Availability Direction Y H BI
Figure 14.
X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Relay Latency 5ms
Jitter 0.2ms
Acknowledge Y
Trans. Rate
Actor 2 Relay Packet Loss Security 0.1% H Time Synch. Y
BER / PER …
Current Differential Protection Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
24
4.1.4 Distance Protection Scheme Distance (Impedance Relay) protection scheme is based on voltage and current measurements. A fault on a circuit will generally create a sag in the voltage level. If the ratio of voltage to current measured at the protection relay terminals, which equates to an impedance element, falls within a set threshold the circuit breaker will operate. The operating characteristics of this protection are based on the line characteristics. This means that when a fault appears on the line, the impedance setting in the relay is compared to the apparent impedance of the line from the relay terminals to the fault. If the relay setting is determined to be below the apparent impedance it is determined that the fault is within the zone of protection. When the transmission line length is under a minimum length distance protection becomes more difficult to coordinate. In these instances the best choice of protection is current differential protection. The schemes may work in different modes (trip, blocking...).
Use Case General Description Title Distance Protection Description Distance protection scheme relies on both voltage and current measurements. If the ration of voltage over current is above a certain threshold (indicating a very small impedance equivalent) the relay will operate and open the breaker to isolate the line. Actors Distance protection relays; Breakers Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
25
Communication Events & Attributes Event Block / Trip Signal Mechanism Protocol Bandwidth EventProprietary 64 kbps triggered Redundancy Availability Direction Y H BI
Figure 15.
Actor 1 Relay Latency 5ms
Jitter 0.2ms
Acknowledge Y
Trans. Rate
Actor 2 Relay Packet Loss Security 0.1% H Time Synch. Y
BER / PER …
Distance Protection Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
26
4.1.5 Inter-Substation Protection Signaling This use case describes the exchange of Sampled Value or GOOSE message between IED’s in two substations for protection and tripping coordination. The two IED’s are in a master-slave mode. The CT/VT in one substation sends the sampled analog voltage or current value to the Merging Unit (MU) over hard wire. The merging unit sends the time-synchronized 61850-9-2 sampled values to the slave IED. The slave IED forwards the information to the Master IED in the other substation. The master IED makes the determination (for example based on sampled value differentials) to send a trip command to the originating IED. Once the slave IED/Relay receives the GOOSE trip for breaker tripping, it opens the breaker. It then sends a confirmation message back to the master. All data exchanges between IEDs are either through Sampled Value or GOOSE messages. The GPS Master Clock can send 1PPS or IRIG-B format to MU through serial port, or IEEE 1588 protocol via network.
Use Case General Description Title Inter-Substation Protection Signaling using 61850 Description Exchange of GOOSE message between IED’s in two substations for protection and tripping coordination. Actors CT/VT; Merging Unit; IED; Time Synchronization Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
27
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction Y H BI Communication Events & Attributes Command Monitor Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction Y H BI
Communication Events & Attributes Event Confirmation Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction Y H BI
Actor 1 IED / Relay Latency Jitter 5ms N/A Acknowledge Y
Trans. Rate
Actor 1 IED / Relay Latency Jitter 5ms N/A Acknowledge Y
Trans. Rate
Actor 1 IED / Relay Latency Jitter 5ms N/A Acknowledge Y
Trans. Rate
Date August 26, 2014
Actor 2 IED / Relay Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 IED / Relay Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 IED / Relay Packet Loss Security 1% H Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
28
Figure 16.
Inter-Substation Protection Signaling with 61850 Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
29
4.1.6 Intra-Substation Process Bus Communication This use case describes the data flow from the CT/VT to the IEDs in the substation via the merging unit (MU). The CT/VT in the substation send the sampled value (analog voltage or current) to the Merging Unit (MU) over hard wire. The merging unit sends the time-synchronized 61850-9-2 sampled values to the IEDs in the substation in GOOSE message format. The GPS Master Clock can send 1PPS or IRIG-B format to MU through serial port, or IEEE 1588 protocol via network. Process bus communication using 61850 simplifies connectivity within the substation and removes the requirement for multiple serial connections and removes the slow serial bus architectures that are typically used. This also ensures increased flexibility and increased speed with the use of multicast messaging between multiple devices.
Use Case General Description Title Intra-Substation 61850 GOOSE Data Flow Description Sampled values collected through CT/VT are sent from the process bus to IEDs on the station bus in the IEC 61850 GOOSE message format. Actors CT/VT; Merging Unit; IED; Time Synchronization Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
30
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction N H BI Communication Events & Attributes Event Command Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction N H BI
Communication Events & Attributes Event Acknowledgement Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction N H BI
Actor 1 IED / Relay Latency Jitter 5ms N/A Acknowledge Y
Trans. Rate
Actor 1 IED / Relay Latency Jitter 5ms N/A Acknowledge Y
Trans. Rate
Actor 1 IED / Relay Latency Jitter 5ms N/A Acknowledge Y
Trans. Rate
Date August 26, 2014
Actor 2 IED / Relay Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 IED / Relay Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 IED / Relay Packet Loss Security 1% H Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
31
Figure 17.
Process Bus Communication using 61850 Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
32
4.1.7 Control Center Monitoring of Station Bus This use case describes the process of polling substation station bus from the EMS or DMS application in the control center where the communication between control center and the substation is via 61850-90-2 and communication within the substation is via 61850.
Use Case General Description Title Control Center Monitoring of Substation Station Bus with 61850 Description Monitoring of substation Station Bus from a control center application, e.g. EMS or DMS. Communication within substation is via IEC 61850 and communication to the control center is via 61850-90-2. Actors Control Center applications (EMS or DMS); IEC 61850 gateway; Station Bus; IED
Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction Y H BI
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Control Center Latency Jitter < 1sec N/A Acknowledge
Trans. Rate
Date August 26, 2014
Actor 2 61850 Gateway Packet Loss Security 1% H Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
33
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Event61850-9-2 64 kbps triggered Redundancy Availability Direction Y H BI
Figure 18.
Actor 1 61850 Gateway Latency Jitter 5ms N/A Acknowledge
Trans. Rate
Actor 2 IED Packet Loss Security 1% H Time Synch. Y
BER / PER …
Control Center Monitoring of Station Bus using 61850 Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
34
4.1.8 Communication System Failure & Degradation Monitoring Due to the critical role of the communication network in the next generation monitoring, control, and protection functions, it is important that the failure and degradation of the communication network is constantly monitored. The IEC 62351-7 standard calls for monitoring the communication network to detect and log the following conditions: • • • • • •
Network equipment permanent failures Network equipment temporary failures or resets Communication link failures Communication link degradation or lower than expected throughput Network routing degradation or lower than expected throughput Logging equipment and communication link failures and degraded conditions
This use case describes the monitoring of the communication network, including device status, device configuration, latency test, software management, and threshold testing by the Network Management System. The benefits include improved visibility of communication network faults; reduced communications minutes lost due to undetected faults, and proactive maintenance of communications assets.
Use Case General Description Title Communication System Failure and Degradation Monitoring Description Monitoring the failure or degradation of the communication system using NMS. Actors Network Management System; Communication Device (Router/Switch)
Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise
Date August 26, 2014
X
X X X X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
35
Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Configuration Mechanism Protocol Bandwidth Ad hoc SSH 64 kbps Redundancy Availability Direction N H BI Communication Events & Attributes Event Threshold Event Mechanism Protocol Bandwidth Ad hoc SNMPv3 9.6 kbps Redundancy Availability Direction N H BI Communication Events & Attributes Event Device Status Mechanism Protocol Bandwidth Ad hoc SNMPv3 9.6 kbps Redundancy Availability Direction N H BI Communication Events & Attributes Event Software Management Mechanism Protocol Bandwidth Ad hoc SCP 64 kbps Redundancy Availability Direction N H BI
Communication Events & Attributes Event Latency Test Mechanism Protocol Bandwidth Ad hoc ICMP 9.6 kbps Redundancy Availability Direction N H BI
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 NMS Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Actor 1 Device Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Actor 1 Device Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Actor 1 NMS Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Actor 1 Device Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Date August 26, 2014
X X
Actor 2 Device Packet Loss Security 5% M Time Synch. BER / PER Y …
Actor 2 NMS Packet Loss Security 1% M Time Synch. BER / PER Y …
Actor 2 NMS Packet Loss Security 5% M Time Synch. BER / PER Y …
Actor 2 Device Packet Loss Security 5% M Time Synch. BER / PER Y …
Actor 2 NMS Packet Loss Security 5% H Time Synch. BER / PER Y … Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
36
Figure 19.
Communication System Failure & Degradation Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
37
4.1.9 High Voltage Substation SCADA RTU The control and supervision of high voltage substations is done via the use of Remote Terminal Units (RTU). Today, most RTU’s use serial connection and proprietary legacy protocols. In this use case we model a generic device to control center communication through the RTU using IEC 60870-5-104 or IEC 60870-5-101 protocols. For high voltage substations communication network availability and performance for SCADA applications are critical.
Use Case General Description Title High Voltage Substation SCADA RTU Using IEC 60870-5-101 & 104. Description Generic use case describing communication between substation RTU and control center using IEC 60870-5-101 & 104. Actors Substation RTU; Control Center SCADA Headend Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-101 20 kbps & 104 Redundancy Availability Direction
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge
Trans. Rate
Date August 26, 2014
Actor 2 RTU Packet Loss Security 5% H Time Synch.
BER / PER
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
38
Y
H
BI
Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Ad hoc 60870-5-101 20 kbps & 104 Redundancy Availability Direction Y H BI Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Polled 60870-5-101 20 kbps & 104 Redundancy Availability Direction Y H BI
Communication Events & Attributes Event Time Synch Mechanism Protocol Bandwidth Ad hoc 60870-5-101 20 kbps & SNTP Redundancy Availability Direction Y H BI
Y
1 / Second or two Seconds
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate 1 / Second or two Seconds
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate
Date August 26, 2014
Y
…
Actor 2 Control Center Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 RTU Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% H Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
39
Figure 20.
High Voltage Substation SCADA RTU Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
40
4.1.10 Medium Voltage Substation SCADA RTU The control and supervision of medium voltage substations are done via the use of Remote Terminal Units (RTU). Today, most RTU’s use serial connection and proprietary legacy protocols. In this use case we model a generic device to control center communication through the RTU using IEC 60870-5-104 or IEC 60870-5-101 protocols. While the bandwidth and latency requirements are similar to those for the high voltage substations, the requirements for uptime and availability are less stringent for medium voltage substation networks.
Use Case General Description Title Medium Voltage Substation SCADA RTU Using IEC 60870-5-101 & 104. Description Generic use case describing communication between substation RTU and control center using IEC 60870-5-101 & 104. Actors Substation RTU; Control Center SCADA Headend Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-101 9.6 kbps & 104
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Control Center Latency Jitter 0.5 Sec N/A
Date August 26, 2014
Actor 2 RTU Packet Loss Security 5% H Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
41
Redundancy Y
Availability H
Direction BI
Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Ad hoc 60870-5-101 9.6 kbps & 104 Redundancy Availability Direction Y H BI
Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Polled 60870-5-101 9.6 kbps & 104 Redundancy Availability Direction Y H BI
Communication Events & Attributes Event Time Synch Mechanism Protocol Bandwidth Ad hoc 60870-5-101 9.6 kbps & SNTP Redundancy Availability Direction Y H BI
Acknowledge Y
Trans. Rate 1 / Second or two Seconds
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate 1 / Second or two Seconds
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate
Date August 26, 2014
Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 RTU Packet Loss Security 1% H Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% H Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
42
Figure 21.
Medium Voltage Substation SCADA RTU Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
43
4.1.11 Condition-Based Monitoring This use case applies to both Transmission and Distribution service categories. T&D Operations can use sensors to proactively monitor equipment in the field to make maintenance decisions based on the current conditions of the assets. Automated analysis is performed on sensor data using rule-based algorithms to identify assets that are potentially in need of repair or replacement. The following are just a few examples of what can be monitored for condition-based maintenance: • • •
Transformer oil temperature and oil pressure monitoring Turbine monitoring Backup battery monitoring
There are many benefits to condition-based monitoring including: improved SAIDI/SAIFI, higher efficiency in operations, improved asset uptime, outage reduction, better system monitoring, increased crew safety, increased public safety, improved power quality, reduced truck rolls, and deferred capital expenditure by extending the useful life of the asset. Condition based monitoring can also be used to automate certain maintenance routines by linking routine inspection and work scheduling to the results of asset condition assessment. It can also alleviate costs associated with mandatory regulatory requirements for visual inspection by recording the periodically collected asset data for auditing purposes. With orders of magnitude increase in the volume of data that are collected utilities will use Big Data query and analysis tools to extract business intelligence and drive higher efficiency in asset utilization.
Use Case General Description Title Condition-Based Monitoring using IEC 60870 Description Online monitoring of field assets’ condition to drive maintenance and repair scheduling, to improve the efficiency of operation, and to improve asset reliability and life expectancy. Actors Substation RTU; Data Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Date August 26, 2014
X X X X Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
44
Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction N M BI Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction N M BI
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 RTU Latency 1 Sec Acknowledge
Jitter N/A Trans. Rate 1 / 4 Hours
Actor 1 Data Center Latency Jitter 1 Sec N/A Acknowledge Trans. Rate
Date August 26, 2014
X X
Actor 2 Data Center Packet Loss Security 5% M Time Synch. BER / PER Y …
Actor 2 RTU Packet Loss Security 1% M Time Synch. BER / PER Y …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
45
Figure 22.
Condition Based Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
46
4.1.12 Transformer Dissolved Gas Analysis Transformers are designed to operate 30-50 years. Periodic analysis of transformer oil provides an indication of the condition of the transformer, which can provide valuable information to substation designers and engineers. The presence or changes in dissolved gas indicates internal changes in the transformer, sometimes due to deterioration of seals between internal components. Today the analysis is done based on manual samples taken every couple of years. This use case describes automated monitoring of dissolved gas every four hours. The analysis typically includes hydrogen (H2), oxygen (O2), nitrogen (N2), methane (CH4), carbon monoxide (CO), carbon dioxide (CO2), ethylene (C2H4), ethane (C2H6), acetylene (C2H2), and propane (C3H8). The data could be transferred to the Data Center via IEC 60870-5-104 or a legacy SCADA protocol such as DNP3, over the IP network. Continuous collection of data on grid assets provides a rich data set that can be mined for optimizing the utilization and life expectancy of the assets. Here is yet another opportunity for using Big Data query and analysis tools to uncover failure patterns that once identified could be preventable by timely maintenance or a shift in operational duty cycles. This analysis can be done for both transmission and distribution transformers.
Use Case General Description Title Transformer Dissolved Gas Analysis using IEC 60870 Description Online monitoring and analysis of transformer dissolved gas analysis to detect anomalies and signs of wear and failure for preventive maintenance. Actors Gas Analysis Sensor; Data Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise
Date August 26, 2014
X X X
X
X X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
47
Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-104 96 kbps or DNP3 over IP Redundancy Availability Direction N M UNI
Figure 23.
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 RTU Latency 1 Min
Jitter N/A
Acknowledge
Trans. Rate 1 / 4 Hours
X
Actor 2 Data Center Packet Loss Security 10% M
Time Synch. Y
BER / PER …
Transformer Dissolved Gas Analysis Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
48
4.1.13 Gas-Insulated Switchgear Health Monitoring Gas-insulated switchgear could be monitored at regular intervals to increase the overall operational efficiency of the asset by reducing the SF6 gas inspection time. Monitoring the real-time gas density value in combination with historical trends for gas leakage rate allows the company to predict and optimize equipment maintenance scheduling, going from time-based to condition-based maintenance. This use case describes automated monitoring of SF6 gas every four hours. The data could be transferred to the Data Center via IEC 60870-5-104 or a legacy SCADA protocol such as DNP3, over the IP network. This is yet another opportunity to use Big Data analytics tools for uncovering pre-failure asset condition patterns from volume of data that is collected on similar assets to proactively predict and prevent asset failures. This analysis can be done for both transmission and distribution switchgear.
Use Case General Description Title Gas-Insulated Switchgear Health Monitoring using IEC 60870 Description Online monitoring and analysis of gas-insulated switchgear to detect anomalies and signs of equipment wear and failure for preventive maintenance purposes. Actors RTU; Data Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
X
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
49
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled DNP3 over 20 kbps IP or 60870-5-104 Redundancy Availability Direction N M UNI
Figure 24.
Actor 1 RTU Latency 1 Min
Jitter N/A
Acknowledge
Trans. Rate 1 / 4 Hours
Actor 2 Data Center Packet Loss Security 5% M
Time Synch. Y
BER / PER …
Gas-Insulated Switchgear Health Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
50
4.1.14 Dynamic Asset Rating Dynamic asset rating refers to the ability to remotely monitor transmission line conditions (with sensors that detect conductor temperature, line sag, and wind speed and direction) to determine the maximum power carrying capacity and loading of the line. This provides a more accurate and timely view of the line capacity compared to static nameplate limits for the line. Armed with this information operators can push more power through the line to alleviate transmission constraints and network congestion issues. This leads to a more effective utilization of the asset capability. It is conceivable that with continuous collection and monitoring of the assets using Big Data Analytics tools the operators will be able to anticipate asset conditions under different grid operating states and automate decisions for line loading for different grid states. Use Case General Description Title Dynamic Asset Rating using IEC 60870 Description The ability to remotely monitor transmission line conditions and determine excess capacity of the line for carrying additional power flow. Actors RTU; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth
X
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Control Center Latency Jitter
Date August 26, 2014
Actor 2 RTU Packet Loss Security Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
51
Polled Redundancy N
60870-5-104 Availability M
20 kbps Direction BI
0.5 Sec Acknowledge Y
N/A Trans. Rate 1 / Min
5% Time Synch. Y
H BER / PER …
Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Ad hoc 60870-5-104 20 kbps Redundancy Availability Direction N M BI
Latency 0.5 Sec Acknowledge Y
Jitter N/A Trans. Rate 1 / Min
Actor 2 Control Center Packet Loss Security 1% H Time Synch. BER / PER Y …
Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction N M BI
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y 1 / Min
Actor 2 RTU Packet Loss Security 1% H Time Synch. BER / PER Y …
Actor 1 RTU
Actor 2 Control Center Packet Loss Security 1% H Time Synch. BER / PER Y …
Communication Events & Attributes Event Time Synch Mechanism Protocol Bandwidth Polled SNTP 20 kbps Redundancy Availability Direction N M BI
Actor 1 RTU
Latency 0.5 Sec Acknowledge Y
Jitter N/A Trans. Rate 1 / Min
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
52
Figure 25.
Dynamic Asset Rating Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
53
4.1.15 Wide Area Monitoring & Control Systems The application of synchrophasor measurement data from Phasor Measurement Units (PMU) to Wide Area Monitoring and Control Systems promises to provide important new capabilities for improving system stability. Access to PMU data enables more timely situational awareness over larger portions of the grid than what has been possible historically with normal SCADA data. Handling the volume and real-time nature of synchrophasor data presents unique challenges for existing application architectures. Wide Area management System (WAMS) makes it possible for the condition of the bulk power system to be observed and understood in real-time so that protective, preventative, or corrective action can be taken. The history of PMU-based WAMS at H-Q goes back more than 30 years to the mid-1970s. In 2004, H-Q commissioned an 8-PMU WAMS system that feed EMS with GPS-synchronized angles, frequencies and harmonic distortion measurements from key 735 KV substations. H-Q uses this system for frequency regulation reporting and control room implementation of preventive measures against geomagnetic storm-induced contingencies. Because of the very high sampling rate of measurements and the strict requirement for time synchronization of the samples, WAMS has stringent communication requirements in an IP network that are captured in the following table:
WAMS Requirement Client Interfaces One Way Delay Delay Symmetry Required Jitter Layer 2 or Layer 3 VPN Topology Multicast Availability / Reliability Precise Timing Required Link / Node Failure Delay Performance Management
Figure 26.
Attribute Ethernet, Serial, X.21 50ms No Not Critical Layer 3 Multi-Point to Multi-Point Layer 3 99.999 / High Yes 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Stream C37.118 100 kbps Redundancy Availability Direction Y H UNI
Latency 100 ms Acknowledge Y
Jitter N/A Trans. Rate 60 / sec
Actor 2 PDC / Control Center Packet Loss Security 1% H Time Synch. BER / PER Y …
Communication Events & Attributes Event Time Sync Mechanism Protocol Bandwidth Polled IEEE 1588 20 kbps Redundancy Availability Direction Y H UNI
Actor 1 PMU / PDC Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y 60 / sec
Actor 2 Control Center Packet Loss Security 1% H Time Synch. BER / PER Y …
Actor 1 PMU
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
55
The following UML diagrams represent three distinct possibilities: 1. Wide Area Management System with PDCs 2. Wide Area Management System without PDCs 3. Wide Area Management System without PDCs and with control capability
Figure 27.
Wide Area Management System with PDC Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
56
Figure 28.
Figure 29.
Wide Area Management System without PDC Graphic Information
Wide Area Management System with P2P Control Capability Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
57
4.1.16 Substation Environmental Monitoring This use case applies to both transmission and distribution substations. Examples of environmental monitoring sensors include: substation internal and external ambient temperature, transformer temperature, battery temperature, humidity sensor, airflow sensor, flood sensors, fuel level, earthquake, etc. Typically, environmental measurements are taken at 10-minute intervals and sent to the control center for archival and analysis. However, depending on the measurement type, samples could be taken less or more frequently, e.g. fuel levels are measured once a week. The benefits of substation environmental monitoring are: • • • • • •
Avoiding equipment damage Real-time prediction of likely outages Reducing the impact of low probability, high impact events, such as flooding Creating more accurate asset lifetime model Developing more accurate transformer rating Making informed decisions about sending field crew to locations that could be intolerably hot or humid
Use Case General Description Title Substation Environmental Monitoring Description Periodic monitoring of substation environmental conditions for better asset management and improved system reliability. Actors Environmental Sensor(s); Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN)
X X X
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network
Date August 26, 2014
X X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
58
Substation Network (Transmission)
X
Residential Premise Network
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled DNP3 9.6 kbps Redundancy Availability Direction N L UNI
Actor 1 Air Flow Sensor Latency Jitter 1 Min N/A Acknowledge Trans. Rate N Variable
Actor 2 Control Center Packet Loss Security 10% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled DNP3 9.6 kbps Redundancy Availability Direction N L UNI
Actor 1 Battery Sensor Latency Jitter 1 Min N/A Acknowledge Trans. Rate N Variable
Actor 2 Control Center Packet Loss Security 10% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled DNP3 9.6 kbps Redundancy Availability Direction N L UNI
Actor 1 Humidity Sensor Latency Jitter 1 Min N/A Acknowledge Trans. Rate N Variable
Actor 2 Control Center Packet Loss Security 10% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled DNP3 9.6 kbps Redundancy Availability Direction N L UNI
Actor 1 Flood Sensor Latency Jitter 1 Min N/A Acknowledge Trans. Rate N Variable
Actor 2 Control Center Packet Loss Security 10% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled DNP3 9.6 kbps Redundancy Availability Direction N L UNI
Actor 1 Temperature Sensor Latency Jitter 1 Min N/A Acknowledge Trans. Rate N Variable
Actor 2 Control Center Packet Loss Security 10% M Time Synch. BER / PER Y …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
59
Figure 30.
Substation Environmental Monitor Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
60
4.1.17 Power System State Estimation This use case describes the data acquisition process for power system State Estimation. RTUs at generation and transmission substations are periodically polled to send analog and status measurements including voltages, currents, phase angles, real and reactive power flows, and breaker and other logical device statuses to the SCADA headend in the Control Center and on to the State Estimator. The State Estimator uses statistical estimation algorithms to determine the topology of the electrical grid, to sort out “bad data”, and to estimate the current status of all flows and devices based on a consistent snapshot of network state measurements at appoint in time. The results of the State Estimator are used by other advanced monitoring and control applications in the Energy Management System to optimize system performance by minimizing system losses. Data exchange between these applications is via in-memory databases in the control center. In many instances RTUs use serial connection and proprietary protocols. In this use case we model RTU to Control Center communication using IEC 60870-5-104 or IEC 60870-5-101 protocols. For transmission substation communication availability and performance of the RTU is critical.
Use Case General Description Title SCADA Data Collection for State Estimation Description Collection and transmission of data from RTUs using IEC 60870-5-104 and 60870-5-101 protocols. Actors Substation RTU; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
61
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 61850-5-101 20 kbps & 61850-5-104 Redundancy Availability Direction Y H BI Communication Events & Attributes Event Alarm Mechanism Protocol Bandwidth Ad hoc 61850-5-101 20 kbps & 61850-5-104 Redundancy Availability Direction Y H BI Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Polled 61850-5-101 20 kbps & 61850-5-104 Redundancy Availability Direction Y H BI Communication Events & Attributes Event Time Sync Mechanism Protocol Bandwidth Polled 61850-5-101 20 kbps & SNTP Redundancy Availability Direction Y H BI
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate Every 2 Sec
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate Every 2 Sec
Actor 1 Control Center Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate …
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate Every 2 Sec
Date August 26, 2014
Actor 2 Control Center Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Actor 2 RTU Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% M Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
62
Figure 31.
State Estimation Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
63
4.1.18 Inter Control Center Communication (ICCP) The Inter-Control Center Communications Protocol (ICCP) or IEC 60870-6 is used for real-time data exchange among utility control centers, regional control centers, and non-utility generators. Through ICCP, utilities periodically exchange system data include analog and status points, quality flags, setpoints, events, and error conditions, etc. This use case describes the process of updating SCADA data in one Control Server (ICCP Client) by periodically (every 4-8 seconds) sending analog and status values representing transmission and distribution network telemetered or calculated values and generation related data from another Control Center (ICCP Server). Occasional non-delivery could be tolerated as long as the consuming applications in the receiving Control Center are made aware of the non-delivery and therefore could reinitialize the database. Otherwise, inconsistent ICCP data could lead to corruption of database in the receiving Control Center. For ICCP, the best practice is to use “Secure ICCP” as outlined in the “Secure ICCP Integration Considerations and Recommendations” by Sandia National Laboratories. The Sandia report’s recommendations are restated here: Secure ICCP Certificate Management •
PKI Domain Design – Based on best-practice implementations, two primary PKI domain designs were identified: a flat hierarchy and a tiered hierarchy. For control systems within a single established domain, a flat hierarchy is recommended for the distribution of authentication certificates. This recommendation is based on the number of endpoints sharing ICCP data. For the most part, such networks tend to be isolated and generally small (at most a few hundred nodes) and, as such, lend themselves better to a flat hierarchy. The advantage of a flat hierarchy is that only one CA needs to be established for everyone on the internal domain network, reducing the complexity of the configuration. In a tiered approach, each company would maintain its own CA, a proposition that is likely costprohibitive and more managerially complex.
•
Inter-Domain Communication – The architecture recommended for inter-domain communication is a tiered hierarchy. This recommendation is based on the need to provide the most secure implementation. Creating a single “root” Certificate Authority (CA) allows more restrictive security policies to be enforced at the root while alleviating some of stringent security requirements on subordinate CA’s.
•
Secure ICCP Application Issues – Current implementations of certificate-based schemes within ICCP applications are primarily static in nature. This implies that any certificate update or renewal process requires actions by an operator. This mechanism does not fit modern techniques of end node authentication. Web-based forms of certificate authentication do not require machines (computers) to be informed of the certificate update because the new certificate will be sent at the beginning of each
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
64
SSL handshake. Because a node’s certificate is sent at the beginning of each session, nodes should not need to store local copies of anyone else’s certificate. Therefore, when a node is issued a new certificate for any reason (expiration, key update, etc.), the operation is transparent to other nodes in the network and they do not need to be notified. It is recommended that these techniques be designed into all applications intended to support Secure ICCP. Network System Design & Quality of Service •
In any Wide Area Network (WAN), the most efficient and highly available routes will become congested and communication between participating end nodes, e.g. SCADA control centers, may be delayed or lost. Therefore, it is essential to create Service-Level Agreements (SLAs) for WAN that guarantee a level of service for ICCP data streams.
Transition Strategy •
Layer 2 and Layer 3 Protection Schemes – For some utility sites the conversion from the standard ICCP to Secure ICCP will not be rapidly achieved. The report discusses some potential alternatives to provide the security needed to assure ICCP data protection. IPSec and data link encryption are suggested as means to provide the necessary data surety for the protection of in-flight ICCP data. A technique is also described to configure a network connection to provide a mixed-mode operational scenario when both secure and non-secure forms of ICCP co-exist on a network.
Performance •
The report discusses measurements that were taken to characterize the impact of using different security layers associated with securing the ICCP data. The processing and transport delays are characterized to provide the user with a sense of the operational impact when adding protection technologies to an ICCP network. Associated implementations, such as OpenSSL for Secure ICCP and IPSec for a Layer-3 encryption, are documented. The overall results show that the integration of secure protocols should have minimal effect on end-to-end application performance but the overall management complexity will increase with each added layer of protection.
Use Case General Description Title Inter Control Center Communication (ICCP) Description SCADA data transfer between two control centers using ICCP protocol Actors Sending Control Center (ICCP Server); Receiving Control Center (ICCP Client) Business Service Category Transmission Distribution Energy Supply Demand Side
X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
65
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Communication Events & Attributes Event SCADA Data Mechanism Protocol Bandwidth Polled IEC 60870-6 > 1MB Redundancy Availability Direction Y H UNI
Actor 1 ICCP Server Latency Jitter < 4 Sec N/A Acknowledge Trans. Rate Y 1 every 4 Sec
Actor 2 ICCP Client Packet Loss Security 5% H Time Synch. BER / PER Y …
Communication Events & Attributes Event Delivery Flag Mechanism Protocol Bandwidth Polled IEC 60870-6 9.6 kbps Redundancy Availability Direction Y H UNI
Actor 1 ICCP Server Latency Jitter < 4 Sec N/A Acknowledge Trans. Rate Y 1 every 4 Sec
Actor 2 ICCP Client Packet Loss Security 5% H Time Synch. BER / PER Y …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
66
Figure 32.
ICCP Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
67
4.1.19 RTU File Transfer All RTU maintain some record files with chronological data. This use case describes communication between a dedicated application in the control center that uploads and downloads database files to and from the RTUs. The RTU file is downloaded for analysis. The application uses the IEC 60870-5-101/104 SCADA protocol to perform this operation. This application is basically an IEC 60870-5-104 file transfer to HTTP converter /gateway with two IP interfaces. One connected to the intranet, and the other to the substation device management network. Once a user opens an http connection with this application, that query is translated into a file transfer using IEC 60870-5-104 against a specific RTU. There are two options: 1. Whole RTU configuration file upload/download 2. Selective, line-by-line update of RTU configuration
Use Case General Description Title RTU File Transfer Description Transferring the RTU configuration file and binary database to / from the RTU to the Control Center. Actors RTU; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
68
Communication Events & Attributes Event File Transfer Mechanism Protocol Bandwidth Ad hoc 60870-5-101 64 kbps & 60870-5-104 Redundancy Availability Direction Y H BI Communication Events & Attributes Event Set Point Update Mechanism Protocol Bandwidth Ad hoc 60870-5-101 9.6 kbps & 60870-5-104 Redundancy Availability Direction Y H BI
Figure 33.
Actor 1 RTU Latency 5ms
Jitter N/A
Acknowledge Y
Trans. Rate …
Actor 1 Control Center Latency Jitter 5ms N/A
Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% H
Time Synch. Y
BER / PER …
Actor 2 RTU Packet Loss Security 5% H
Time Synch. Y
BER / PER …
RTU File Transfer Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
69
4.1.20 Weather Monitoring Stations By monitoring weather in real-time the utility will be able to predict weather-sensitive load. Also, advance warning of severe weather conditions can prepare the utility to cope with potential outages that are caused by extreme weather conditions. Another advantage of having real-time weather data in the vicinity of transmission lines is to be able to calculate their dynamic ratings. And finally, weather stations can provide valuable data on wind speed and direction to forecast wind farm generation. For this reasons many utilities install their own weather stations to monitor local weather conditions. The weather station acts as a sensor that can measure and forward analog and digital data to the control center through standard SCADA protocols, either over IP or a legacy protocol such as DNP3.
Use Case General Description Title Weather Monitoring Stations Description Collecting local weather data via utility-owned weather stations for load forecasting, dynamic asset rating calculation, and asset condition monitoring. Actors Weather Sensor; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled DNP3 9.6 kbps
X X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Weather Sensor Latency Jitter N/A N/A
Date August 26, 2014
X
Actor 2 Control Center Packet Loss Security 5% M Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
70
Redundancy N
Availability L
Figure 34.
Direction UNI
Acknowledge N
Trans. Rate Once an Hour
Time Synch. Y
BER / PER …
Weather Monitoring Station Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
71
4.2 Distribution This section includes use cases related to power distribution including distribution and feeder automation and centralized grid monitoring and control of the distribution grid through control center applications such as SCADA, DMS, and OMS.
4.2.1 Power Quality Monitoring System – PQMS This use case describes the installation and operation of a unified and centralized power quality monitoring system to enable the utility to meet increasing demand for power quality benchmarking, power quality contracts, billing and energy use verification, predictive maintenance, etc. It is assumed that a centralized head end server communicates with various sensors, including IEDs, meters, and protective relays that collect the relevant data at selected locations across the grid. Information from the sensors is retrieved periodically, or by exception if there are unexpected events, by PQMS. This information is stored for archival and reporting purposes and is made available to various applications for analysis. Here are some of the metrics that are used in power quality monitoring. The IEEE 1159 standard defines these metrics as follows: • • • • • • • • •
Flicker – Impression of unsteadiness of visual sensation induced by a light stimulus whose luminance or spectral distribution fluctuates with time. Voltage or Current Imbalance – The ratio of the negative sequence component to the positive sequence component, usually expressed as a percentage. Momentary Interruption – A type of short-duration root-mean-square (RMS) voltage variation where the complete loss of voltage ( 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
X X
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
75
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled or 60870-5-104 > 9.6 kbps EventTriggered Redundancy Availability Direction N L UNI
Figure 36.
Actor 1 IED / Relay Latency Jitter 5ms N/A
Acknowledge N
Trans. Rate Variable
Actor 2 Control Center Packet Loss Security 5% L
Time Synch. Y
BER / PER …
Sensors in the Distribution Grid Tabular & Graphic Information
4.2.3 Pole-Top Voltage Regulators & Capacitor Bank Monitoring Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
76
Pole top voltage regulators and capacitor banks exist today but are not centrally monitored or controlled during normal system activity. It is desirable to be able to centrally monitor these devices as part of active voltage support and power factor correction activities. Centralized monitoring will enable integrated decision making for optimal device configuration between multiple devices on the same feeder. With the volume of data that will be available, operators will be able to detect grid operating states and dynamically control reactive resources in contrast to traditional way of activating these resources on a fixed preprogrammed schedule. With the increasing volume of data that are collected about voltage regulators and capacitor banks utilities will be able to use Big Data query and analysis tools to extract business intelligence and drive higher efficiency in the utilization of these assets.
Use Case General Description Title Pole Top Voltage Regulator Capacitor Bank Monitoring Description Centralized monitoring of pole top voltage regulator and capacitor bank monitoring for coordinated voltage support and power factor correction decision-making. Actors IED; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 IED / Relay Latency Jitter
Date August 26, 2014
X
Actor 2 Control Center Packet Loss Security Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
77
Polled Redundancy N
60870-5-104 Availability L
Figure 37.
20 kbps Direction UNI
1 Sec Acknowledge Y
N/A Trans. Rate SCADA Scan Rate
5% Time Synch. Y
L BER / PER …
Voltage Regulator / Capacitor Bank Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
78
4.2.4 Remote Control of Overhead or Underground Switches Deployment of remote control overhead switches is a component of the broader fault detection, isolation and service restoration (FLISR) initiative in utilities. Of course there are other components in FLISR, including smart relays, smart sectionalizers / reclosers, real-time distribution feeder monitors, and smart meters. Switching activities are more efficient when they are controlled centrally. These components, along with various monitoring and control algorithms that run in the distribution control center as part of an advanced distribution management system make it possible to detect and respond to faults and restore service in a short time. Overhead switches are largely deployed in rural areas whereas underground switches are deployed in urban areas where feeders are normally underground.
Use Case General Description Title Remote Control of Overhead or Underground Switches Description Remote control of distribution switches from centralized application such as DMS enables faster response to faults and therefore shorter time to service restoration. Actors IED; Control Center
Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
X
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
79
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction y M BI
Latency 0.5 Sec Acknowledge Y
Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Pushed 60870-5-104 20 kbps Redundancy Availability Direction y M BI
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Figure 38.
Actor 1 IED Jitter N/A Trans. Rate 1 / 2 Seconds
Actor 2 Control Center Packet Loss Security 1% L Time Synch. BER / PER Y …
Actor 2 IED Packet Loss Security 1% M Time Synch. BER / PER Y …
Remote Control of Distribution Switches Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
80
4.2.5 Distribution Feeder Voltage Regulation Voltage regulation requires that voltage sensors be placed on the distribution feeder at critical points to monitor the system voltage. In the event of a voltage deviation from a pre-specified band, the voltage sensors will send an alarm back to the control center. This may then require a scan of the voltage sensors or measurements from the AMI meters and other relevant load meters in the affected area to compute an optimal voltage solution. Commands could then be sent to voltage regulators (tap changers) to move up or down accordingly. The voltage regulator accepts the new control setpoints and gradually drives the voltage back to the acceptable normal band. The continuous flow of data from the distribution grid and smart meters and subsequent processing of that data for optimal voltage profile calculation demand scalable Field Area Network and Wide Area Network and the requisite processing power in the Control Center for timely analysis of that information and issuing the necessary control commands. Furthermore, if there are controllable distributed energy resources or EV charging stations on the feeder, the DMS could send new setpoints to the affected devices. The Voltage regulators play a key part in Volt/VAR control, stabilizing and flattening feeder voltages, thereby reducing grid losses.
Use Case General Description Title Distribution Feeder Voltage Regulation Description Monitoring and regulating voltage in distribution grid using voltage regulators and other controllable devices, such as distributed energy resources and electric vehicles and chargers. Actors Voltage sensors; IED; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise
X X
X X
Places in the Communication Network Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
81
Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60780-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction Y M BI Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Ad hoc 60780-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction Y M BI Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Polled 60780-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction Y M BI
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Voltage Sensor Latency Jitter 1 Sec 5 Min
Acknowledge Y
Trans. Rate 1 / Minute
Actor 1 Voltage Sensor Latency Jitter 1 Sec 5 Min
Acknowledge Y
Trans. Rate 1 / Minute
Actor 1 Control Center Latency Jitter 0.5 Sec 5 Min
Acknowledge Y
Trans. Rate …
Date August 26, 2014
X
Actor 2 Control Center Packet Loss Security 5% M
Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Actor 2 IED Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
82
Figure 39.
Distribution Feeder Voltage Regulation Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
83
4.2.6 Distribution Volt / VAR Optimization Coordinated control of voltage and reactive power is a key requirement within the power system. By reducing the amount of reactive power (VARs) flowing on the distribution feeder, the utility can reduce electrical losses and improve the voltage profile along the feeder. The use of capacitor banks and voltage regulators is one way that utilities can control demand and increase system efficiency. Voltage regulating devices are usually installed at the substations and on the feeders. The substation transformers can have tap changers, which can adjust the feeder voltage at the substation, depending on the loading condition of the feeders. Special transformers with load tap changers (LTC) called voltage regulators are also installed at various locations on the feeders, providing fine-tuning capability for voltage at specific points on the feeders. Reactive compensation devices (i.e., capacitor banks) could be located in the substation or on the feeders. Capacitor banks can be fixed or switched. The DMS application in the Control Center monitors real-time voltages, real and reactive power from LTCs, regulators, capacitors, medium voltage sensors, and additional monitoring points, such as customer meters. Using this real time set of analog measurements, the application can minimize the operational costs by managing real time power factor and voltages as close as possible to the substation power factor and desired voltage targets. Today at Hydro Québec, voltage regulation equipment is implemented within some substations. All voltage regulation is locally controlled and the substation and reaction times of 30 seconds are typical. In some substations, this is done manually only. Today, no substations have voltage regulation controlled via the SCADA/DMS.
Use Case General Description Title Distribution Volt / VAR Optimization Description Coordinated monitoring and control of voltage regulators and capacitor banks from DMS to control the power factor and minimize losses and therefore reduce costs. Actors Voltage Sensors; Voltage Regulators; Capacitor Banks; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Date August 26, 2014
X
X X Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
84
Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction N M BI
Actor 1 Voltage Sensor Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Control Center Packet Loss Security 5% L Time Synch. BER / PER Y …
Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Ad hoc 60870-5-104 20 kbps Redundancy Availability Direction N M BI
Actor 1 Voltage Sensor Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Control Center Packet Loss Security 1% L Time Synch. BER / PER Y …
Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction N M BI
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 IED Packet Loss Security 1% M Time Synch. BER / PER Y …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
85
Figure 40.
Distribution Volt/VAR Optimization Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
86
4.2.7
Substation Capacitor Monitoring & Control
This use case applies to both Transmission and Distribution network and applies the monitoring and control of substation capacitor banks from the Control Center. It is assumed that the voltage is measured by an IED at the substation that also provides a means of remotely controlling the capacitor bank upon receiving a command from the control center.
Use Case General Description Title Substation Capacitor Monitoring & Control Description Remote monitoring and controlling of capacitor banks in the substation from the Control Center. Actors IED; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction Y H BI
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 IED / Voltage Sensor Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Date August 26, 2014
Actor 2 Control Center Packet Loss Security 1% M Time Synch. BER / PER Y … Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
87
Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Ad hoc 60870-5-104 20 kbps Redundancy Availability Direction Y H BI
Actor 1 IED / Voltage Sensor Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Control Center Packet Loss Security 1% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Pushed 60870-5-104 20 kbps Redundancy Availability Direction Y H BI
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 IED Packet Loss Security 1% M Time Synch. BER / PER Y …
Figure 41.
Substation Capacitor Monitoring & Control Tabular & Graphic Information
4.2.8 Underground Cable Distributed Temperature Monitoring Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
88
One method of measuring temperature in an underground oil-field cable involves emitting a laser into a fiber optic cable that is laid down in the immediate vicinity of the electrical cable, to measure the temperature of the fiber to very high spatial resolution and thermal accuracy. The fiber optic cable used for this purpose is passive in nature and has no individual sensing point. The monitoring equipment is calibrated by defining thermal sections with similar configuration and environmental conditions including surface elements and fill type. The monitoring instrument measures the temperature based on the Raman effect by analyzing a laser pulse that has propagated through the fiber. The temperature of the fiber is determined by measuring the Raman scattering of the light. The position of the temperature reading is determined by measuring the arrival timing of the returning light pulse, similar to a radio echo. The temperature limit is translated into a current (loading) limit based on conservative assumptions of the cable configuration and the thermal resistivity of the soil, that is how quickly heat generated by the current in the cable can be dissipated. The benefits of monitoring cable temperature and current limits include: • • • •
Evaluating the real-time cable rating and therefore improving cable utilization Identifying emerging issues along the cable Improving network reliability through early detection of failures and hotspots Saving costs associated with maintenance and repair of underground cables
Use Case General Description Title Underground Cable Distributed Temperature Monitoring Description Monitoring the temperature as a proxy for current along an underground cable using a fiber optic cable laid in close proximity of the energized electrical cable. Actors Distributed Temperature Sensor (DTS); Data Center Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise
Places in the Communication Network Control Center Data Center Wide Area Network (WAN)
X X X
X
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
89
Substation Network (Transmission)
Residential Premise Network
Communication Events & Attributes Event Temperature Sample Mechanism Protocol Bandwidth Polled HTTPS or 128 kbps DNP3 over IP Redundancy Availability Direction N L UNI Communication Events & Attributes Event Current Estimate Mechanism Protocol Bandwidth Polled HTTPS or 20 kbps DNP3 over IP Redundancy Availability Direction N L UNI Communication Events & Attributes Event Alarm Mechanism Protocol Bandwidth Ad hoc HTTPS or 20 kbps DNP3 over IP Redundancy Availability Direction N L UNI
Actor 1 DTS Collector Latency Jitter 5 Sec N/A
Acknowledge Y
Trans. Rate …
Actor 1 DTS Collector Latency Jitter 5 Sec N/A
Acknowledge Y
Trans. Rate …
Actor 1 DTS Collector Latency Jitter 5 Sec N/A
Acknowledge Y
Trans. Rate …
Date August 26, 2014
Actor 2 Data Center Packet Loss Security 5% L
Time Synch. Y
BER / PER …
Actor 2 Data Center Packet Loss Security 5% L
Time Synch. Y
BER / PER …
Actor 2 Data Center Packet Loss Security 5% L
Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
90
Figure 42.
Underground Cable Distributed Temperature Sensing Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
91
4.2.9 Online Transformer Condition Monitoring T&D Operations can proactively monitor equipment in the field to make maintenance decisions based on the current condition of the assets. Monitoring the cooling oil, especially as transformers age, can provide critical early warning of transformer issues. If problems are identified early, preventative measures can be taken to prevent a likely failure and significant outages. Automated analysis is performed on sensor data using rule-based algorithms to identify assets that are potentially in need of repair or replacement. This use case describes the ability to monitor transformers in real-time for various reasons, e.g. to drive them to their rated limits and beyond if necessary to meet the load. At the same time, it can warn the operators if a transformer has been operating at or above its nominal operating limit for a prolonged period of time, which increases the risk of asset failure. Therefore, this could serve as a real-time operational advisory tool to increase the distribution network’s reliability and continuity of service. We should point out that this use case applies to both Transmission and Distribution.
Use Case General Description Title Online Transformer Condition Monitoring Description Real-time decision analysis based on the current condition of a transformer to drive it beyond its rated limit or to avoid transformer failure due to operational stress. Actors RTU (Transformer Sensors); Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
X
X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
92
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-101 64 kbps or 60870-5-104 Redundancy Availability Direction N L UNI
Figure 43.
Actor 1 IED / Relay Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate 1 / 5 Minutes
Actor 2 IED / Relay Packet Loss Security 5% L
Time Synch. Y
BER / PER …
Online Transformer Condition Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
93
4.2.10 Switchgear & Transformer Partial Discharge Monitoring It has been observed that transformers and switchgear at sub-transmission and distribution voltage levels typically exhibit increased partial discharge activity prior to causing a fault that activates protection schemes. Transformer bushings can also exhibit increased partial discharge activity prior to failure. Continuously monitoring partial discharge, combined with other health monitoring measures, enables predictive fault diagnostics and can keep the risk of failure and workplace safety under control. The advantages of online continuous partial discharge testing over periodic partial discharge testing are: • • •
Interval-based testing could miss PD activities since PD varies by time. Continuous monitoring overcomes this inherent flaw. Online monitoring is superior to off-line testing since it is done under real operating conditions. Continuous online monitoring reduces maintenance labor costs.
Use Case General Description Title Switchgear & Transformer Partial Discharge (PD) Monitoring Description Continuous online switchgear & transformer partial discharge monitoring to maximize the uptime and reliability of switchgear and transformers. Actors PD Monitor; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
X
X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
X
Communication Events & Attributes Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
94
Mechanism Polled
Redundancy N
Event Monitor Protocol 60870-5-101 or 60870-5-104 Availability L
Figure 44.
Bandwidth 64 kbps
Direction UNI
Actor 1 PD Sensor Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate 1 / Minutes
Actor 2 Control Center Packet Loss Security 5% L
Time Synch. Y
BER / PER …
Partial Discharge Monitoring Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
95
4.2.11 Fault Location Isolation & Service Restoration (FLISR) As the name implies, Fault Location, Isolation, and Service Restoration (FLISR) refers to the ability to automatically locate the fault, isolate the fault, and restore service in the distribution network. It is a “selfhealing” feature whose purpose is to minimize the impact of faults by serving portions of the loads on the affected circuit by switching to other circuits. It reduces the number of customers that experience a sustained power outage by reconfiguring distribution circuits. This will likely be the first wide spread application of distributed intelligence in the grid. Secondary substations can be connected to multiple primary substations. Normally, static power switch statuses (open/closed) in the network dictate the power flow to secondary substations. Reconfiguring the network in the event of a fault is typically done manually on site to operate switchgear to energize/de-energize alternate paths. Automating the operation of substation switchgear allows the utility to have a more dynamic network where the flow of power can be altered under fault conditions but also during times of peak load. It allows the utility to ‘shift’ peak loads around the network. Or, to be more precise, alters the configuration of the network to move loads between different primary substations. The FLISR capability can be enabled in two modes: • •
Managed centrally from DMS, or Executed locally through distributed control via intelligent switches and fault sensors.
There are 3 distinct sub-functions that are performed:
Figure 45.
Three Steps in FLISR
1. Fault Location Identification This sub-function is initiated by SCADA inputs, such as lockouts, fault indications/location, and, also, by input from the Outage Management System (OMS), and in the future by inputs from fault-predicting devices. It determines the specific protective device, which has cleared the sustained fault, identifies the de-energized sections, and estimates the probable location of the actual or the expected fault. It Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
96
distinguishes faults cleared by controllable protective devices from those cleared by fuses, and identifies momentary outages and inrush/cold load pick-up currents. This step is also referred to as Fault Detection Classification & Location (FDCL). This step helps to expedite the restoration of faulted sections through fast fault location identification and improved diagnostic information available for crew dispatch. Also provides visualization of fault information to design & implement a switching plan to isolate the fault. 2. Fault Type Determination I.
Indicates faults cleared by controllable protective devices by distinguishing between: a. Faults cleared by fuses b. Momentary outages c. Inrush/cold load current
II.
Determines the faulted sections based on SCADA fault indications and protection lockout signals
III.
Increases the accuracy of the fault location estimation based on SCADA fault current measurements and real-time fault analysis
3. Fault Isolation & Service Restoration Once the location and type of the fault has been pinpointed the systems will attempt to isolate the fault and restore the non-faulted section of the network. This can have three modes of operation: I.
Closed-loop mode – This is initiated by the Fault location sub-function. It generates a switching order (i.e., sequence of switching) for the remotely controlled switching devices to isolate the faulted section, and restore service to the non-faulted sections. The switching order is automatically executed via SCADA.
II.
Advisory mode – This is initiated by the Fault location sub-function. It generates a switching order for remotely and manually controlled switching devices to isolate the faulted section, and restore service to the non-faulted sections. The switching order is presented to operator for approval and execution
III.
Study mode – the operator initiates this function. It analyzes a saved case modified by the operator, and generates a switching order under the operating conditions specified by the operator.
With the increasing volume of data that are collected through fault sensors utilities will be to use Big Data query and analysis tools to study outage information to anticipate and prevent outages by detecting failure patterns and their correlation with asset age, type, load profiles, time of day, weather conditions, and other conditions to discover conditions that lead to faults and take the necessary preventive and corrective measures. Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
97
Use Case General Description Title Fault Location, Isolation, and Service Restoration (FLISR) Description FLISR is a self-healing feature enabled by sensors, automated switches, and knowledge of the power distribution in a local area thus enabling reconfiguration of the feeders to minimize the number of impacted customers after a fault. Actors Actors Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
Communication Events & Attributes Event Measurement Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction N M BI Communication Events & Attributes Event Fault Event Mechanism Protocol Bandwidth Ad hoc 60870-5-104 20 kbps Redundancy Availability Direction N M BI
X
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 RTU Latency 0.5 Sec Acknowledge Y
Jitter N/A Trans. Rate 1 / sec
Actor 1 RTU Latency 0.5 Sec Acknowledge Y
Jitter N/A Trans. Rate 1 / sec
Date August 26, 2014
X
Actor 2 Control Center Packet Loss Security 1% H Time Synch. BER / PER Y …
Actor 2 Control Center Packet Loss Security 1% H Time Synch. BER / PER Y … Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
98
Communication Events & Attributes Event Switch Control Mechanism Protocol Bandwidth Polled 60870-5-104 20 kbps Redundancy Availability Direction N M BI
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y 1 / sec
Actor 2 RTU Packet Loss Security 1% H Time Synch. BER / PER Y …
Communication Events & Attributes Event Fault Event Mechanism Protocol Bandwidth Ad hoc 61850 64 kbps Redundancy Availability Direction Y H BI
Actor 1 Fault Sensor Latency Jitter 80mSec 40mSec Acknowledge Trans. Rate Y …
Actor 2 Protection Relay Packet Loss Security 0.1% H Time Synch. BER / PER Y …
Figure 46.
Fault Location, Isolation & Service Restoration Tabular & Graphic Information
4.2.12 Voltage Regulation
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
99
A voltage regulator is a transformer with a tap changer that can increase or decrease the voltage on a distribution circuit to help keep the voltage within a pre-determined band. They typically monitor the voltage at the location where they are connected, comparing it to a pre-programmed set point. If the voltage deviates too far from the set point, the voltage regulator can adjust its output voltage by moving the tap on the secondary side up or down. Voltage regulators play a key component in Volt/VAR control, stabilizing and flattening feeder voltages, thereby reducing grid losses. This use case applies to both transmission and distribution.
Use Case General Description Title Voltage Regulation Description Voltage regulation in distribution network using LTC transformers. LTCs can change tap under load thereby changing the voltage on the secondary windings up or down as desired. This use case applies to transmission network as well. Actors Voltage Sensor; Voltage Regulator; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
Communication Events & Attributes Event Monitor
X X X
X
X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Voltage Monitor
Date August 26, 2014
X
Actor 2 Control Center Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
100
Mechanism Polled
Redundancy Y
Protocol 60870-5-101 & 60870-5-104 Availability M
Bandwidth 20 kbps
Latency 0.5 Sec
Jitter N/A
Packet Loss 1%
Security M
Direction UNI
Acknowledge Y
Trans. Rate 1 / 2 Seconds
Time Synch. Y
BER / PER …
Communication Events & Attributes Event Event Mechanism Protocol Bandwidth Ad hoc 60870-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction Y M UNI Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Ad hoc 60870-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction Y M UNI
Actor 1 Voltage Monitor Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate 1 / 2 Seconds
Actor 1 Control Center Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate 1 / 2 Seconds
Date August 26, 2014
Actor 2 Control Center Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Actor 2 Voltage Regulator Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
101
Figure 47.
Voltage Regulation Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
102
4.2.13 Advanced Metering Infrastructure (AMI) This use case describes the implementation of AMI based on open industry standards. AMI refers to the collection of systems that measure, collect, and analyze energy usage and interact with end-point devices such as electric meters, gas meters, and water meters. A typical AMI system records customer consumption at least once an hour and transmits those measurements at least once a day. AMI requires a fixed communication network with stationary transmitters and receivers and must provide two-way communications. Here are some of the AMI benefits: • • • • • • • • •
Provide real-time and detailed information on customer consumption Enable better customer communication and outage notification Provide an opportunity to offer time-based rates Improve reliability and accuracy in meter reading Improve visibility with embedded network diagnostic capabilities Remote connect and disconnect of power Over the air firmware updates and upgrades for meter registration and communication Protect revenue Facilitate customer conservation
In this use case we assume the implementation of a wireless IEEE 802.15.4g/e mesh neighborhood area network (NAN) based on the 6LoWPAN adaptation layer, IPv6 and IPv6 Routing Protocol for Low Power and Lossy Networks (RPL) standard to integrate the NAN into end-to-end network architecture. The network requirements at a high level are: • • • • • •
Device authentication so that no access is granted to unknown devices Encryption on all traffic Prioritization of critical data when sharing the WAN with more critical Distribution Automation traffic Network management for zero-touch provisioning Data integrity, confidentiality, and privacy across the FAN Strong authentication
Use Case General Description Title Advanced Metering Infrastructure Description Automated periodic measurement of end user energy usage along with 2-way communication with remote connects / disconnect capability. Actors Smart meter; AMI Concentrator; Data Center Business Service Category Transmission Distribution
Mobility & Collaboration Physical Safety & Security
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
103
Energy Supply Demand Side
Other (Specify) X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
Communication Events & Attributes Event Meter Data Mechanism Protocol Bandwidth Polled ANSI C12 / 20 kbps / Meter IEEE 1377 Redundancy Availability Direction Y H BI Communication Events & Attributes Event Meter Data Mechanism Protocol Bandwidth Polled ANSI C12 / 20 kbps / Meter IEEE 1377 Redundancy Availability Direction Y H BI Communication Events & Attributes Event Command Mechanism Protocol Bandwidth Polled ANSI C12 / 20 kbps / Meter IEEE 1377 Redundancy Availability Direction Y H BI
X
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Smart Meter Latency Jitter 1 Min … Acknowledge Y
Trans. Rate 1 / Day
Actor 1 AMI Concentrator Latency Jitter 1 Min … Acknowledge Y
Trans. Rate 1 / Day
Actor 1 Data Center Latency Jitter 5 Sec … Acknowledge Y
Trans. Rate 1 / Day
Date August 26, 2014
X X X
Actor 2 AMI Concentrator Packet Loss Security 1% H Time Synch. Y/N
BER / PER …
Actor 2 Data Center Packet Loss Security 1% H Time Synch. Y/N
BER / PER …
Actor 2 AMI Concentrator Packet Loss Security 1% H Time Synch. Y/N
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
104
Communication Events & Attributes Event Command Mechanism Protocol Bandwidth Polled ANSI C12 / 20 kbps / Meter IEEE 1377 Redundancy Availability Direction Y H BI
Figure 48.
Actor 1 AMI Concentrator Latency Jitter 5 Sec … Acknowledge Y
Trans. Rate 1 / Day
Actor 2 Smart meter Packet Loss Security 1% H Time Synch. Y/N
BER / PER …
AMI Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
105
4.2.14 Integrated Grid-Scale Energy Storage This use case is relevant to Transmission, Distribution, and Energy Supply categories. Energy storage technologies have been used by utilities to balance the variability of renewable generation and to reinforce the electricity grid, enabling it to accommodate the increase in peak demands without the need to invest in new generation or T&D expansion. Grid-scale energy storage technologies include pumped hydro • Batteries • Flywheels • Superconducting magnetic energy storage • Ultra-capacitors • Compressed air storage, and • Aggregated plug-in electric vehicles The benefits of grid-scale energy storage include: • • • • •
DMS-controllable devices in the distribution grid Used as Demand Response instruments to shave or shift peak load Used as dispatchable resources inside a microgrid to balance local supply and demand in a local context Used as voltage and reactive power support instruments In aggregate, these devices can provide both real and reactive power support at the transmission level
However, the use of these resources requires real-time communications for monitoring and control purposes. Use Case General Description Title Integrated Grid-Scale Energy Storage Description Grid-scale energy storage resources are increasingly used by utilities in both transmission and distribution grid for a variety of reasons including renewable integration, Demand Response, and voltage and reactive power support. Actors Energy Storage Controller; Control Center; IED (current/voltage/frequency sensors) Business Service Category Transmission Distribution Energy Supply Demand Side
X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Date August 26, 2014
X X X X Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
106
Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 60870-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction N L BI Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Event 60870-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction N L BI Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Event 60870-5-101 20 kbps & 60870-5-104 Redundancy Availability Direction N L BI
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 IED Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate 1 / 2 Seconds
Actor 1 IED Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate …
Actor 1 Control Center Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate 1 / 2 Seconds
Date August 26, 2014
X X
Actor 2 Control Center Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Actor 2 Energy Storage Controller Packet Loss Security 1% M
Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
107
Figure 49.
Integrated Grid-Scale Energy Storage Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
108
4.3 Energy Supply Use Cases This section includes use cases related to centralized power plants, grid scale storage, and distributed generation and storage resources.
4.3.1 Frequency Control / Automatic Generation Control (AGC) The system frequency should be maintained within a very narrow band. Deviations from the acceptable frequency range are detected and forwarded to the Load Frequency Control (LFC) system so that required up or down generation increase / decrease pulses can be sent to the power plants for frequency regulation. The trend in system frequency is a measure of mismatch between demand and generation, and is a necessary parameter for load control in interconnected systems. Automatic generation control (AGC) is a system for adjusting the power output of generators at different power plants, in response to changes in the load. Since a power grid requires that generation and load closely balance moment by moment, frequent adjustments to the output of generators are necessary. The balance can be judged by measuring the system frequency; if it is increasing, more power is being generated than used, and all machines in the system are accelerating. If the system frequency is decreasing, more demand is on the system than the instantaneous generation can provide, and all generators are slowing down. Where the grid has tie lines to adjacent control areas, automatic generation control helps maintain the power interchanges over the tie lines at the scheduled levels. The AGC takes into account various parameters including the most economical units to adjust, the coordination of thermal, hydroelectric, and other generation types, and even constraints related to the stability of the system and capacity of interconnections to other power grids. For the purpose of AGC we use static frequency measurements and averaging methods are used to get a more precise measure of system frequency in steady-state conditions. During disturbances, more real-time dynamic measurements of system frequency are taken using PMUs, especially when different areas of the system exhibit different frequencies. But that is outside the scope of this use case.
Use Case General Description Title Frequency Control / Automatic Generation Control (AGC) Description AGC maintains close balance between total load and total generation in a control area by tracking system frequency as a measure of load-generation imbalance and by sending control signals to power plants to raise or lower their output accordingly. Actors Frequency Sensor; Generation Plant RTU; Control Center Business Service Category Transmission
Mobility & Collaboration
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
109
Distribution Energy Supply Demand Side
Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
Communication Events & Attributes Event Frequency Value Mechanism Protocol Bandwidth Event… 20 kbps triggered Redundancy Availability Direction Y M BI Communication Events & Attributes Event Generator Output Mechanism Protocol Bandwidth Event60870-5-101 20 kbps triggered & 60870-5-104 Redundancy Availability Direction Y M BI Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Event60870-5-101 20 kbps triggered & 60870-5-104 Redundancy Availability Direction
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 PMU Latency 1 Sec
Jitter N/A
Acknowledge Y
Trans. Rate …
Actor 1 Generation Plant RTU Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate 1 / sec
Actor 1 Generation Plant RTU Latency Jitter 0.5 Sec N/A
Acknowledge
Trans. Rate
Date August 26, 2014
X
Actor 2 Control Center Packet Loss Security 5% M Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 5% H
Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 1% H
Time Synch.
BER / PER
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
110
Y
M
BI
Communication Events & Attributes Event AGC Control Mechanism Protocol Bandwidth Event60870-5-101 20 kbps triggered & 60870-5-104 Redundancy Availability Direction Y M BI
Figure 50.
Y
1 / sec
Actor 1 Generation Plant RTU Latency Jitter 0.5 Sec N/A
Acknowledge Y
Trans. Rate 1 / sec
Y
…
Actor 2 Control Center Packet Loss Security 1% H
Time Synch. Y
BER / PER …
Automatic Generation Control Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
111
4.3.2 Hydroelectric Power Plants – General Telemetry There are many parameters in the hydro power generation cycle that are monitored for safety and reliability operation of the plant and the power system. In this section we review the general communication requirements for a hydro plant. In the following sections we will review some of the more specific parameters and their communication requirements. The following is a list of some of the specific parameters that are monitored: • • • • • • •
Oscillographs to detect earth movements Turbine vibration sensors Dam Control Seismometer Security Cameras Water level surveillance Surveillance of the level of ice over the dam using limnimeters
Use Case General Description Title Hydroelectric Power Plants general Telemetry Description General pattern of telemetry for a hydroelectric plant. Actors Plant RTU; Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Date August 26, 2014
X
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
112
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 61850-7-410 20 kbps Redundancy Availability Direction Y H BI
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y 1 / 2 Seconds
Actor 2 Control Center Packet Loss Security 5% L Time Synch. BER / PER Y …
Communication Events & Attributes Event Alert Mechanism Protocol Bandwidth Ad hoc 61850-7-410 20 kbps Redundancy Availability Direction Y H BI
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y 1 / 2 Seconds
Actor 2 Control Center Packet Loss Security 1% L Time Synch. BER / PER Y …
Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Pushed 61850-7-410 20 kbps Redundancy Availability Direction Y H BI
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y 1 / 2 Seconds
Actor 2 Plant RTU Packet Loss Security 1% M Time Synch. BER / PER Y …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
113
Figure 51.
Hydro Plant General telemetry Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
114
4.3.3 Hydro Plant Dam Leakage Monitoring This use case describes the dam leakage monitoring telemetry requirements.
Communication Events & Attributes Event Dam Water Level Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI Communication Events & Attributes Event Alarm Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% H Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 5% H Time Synch. Y
BER / PER …
All other attributes are the same as for the hydro plant general telemetry use case.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
115
Figure 52.
Hydro Plant Dam leakage Supervision Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
116
4.3.4 Hydro Plant Gate Position Indicator This use case describes the gate position indicator telemetry requirements.
Communication Events & Attributes Event Gate Position Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI Communication Events & Attributes Event Upper- or Lower-End Position Reached Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
All other attributes are the same as for the hydro plant general telemetry use case.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
117
Figure 53.
Hydro Plant Gate Position Indicator Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
118
4.3.5 Hydro Plant Water Flow Control This use case describes the water flow control telemetry requirements.
Communication Events & Attributes Event Water Flow Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI Communication Events & Attributes Event Maximum / Minimum Flow Reached Indicator Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
All other attributes are the same as for the hydro plant general telemetry use case.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
119
Figure 54.
Hydro Plant Water Flow Control Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
120
4.3.6 Hydro Plant Water Level Indicator This use case describes the water level indicator telemetry requirements.
Communication Events & Attributes Event Water Level Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI Communication Events & Attributes Event Measuring Device Failure Alarm Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
Actor 2 Control Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
All other attributes are the same as for the hydro plant general telemetry use case.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
121
Figure 55.
Hydro Plant Water Level Indicator Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
122
4.3.7 Hydro Plant Dam Over-Topping Protection This use case describes the dam over-topping protection telemetry requirements. If over-topping is detected the protection will open one or more of the controllable gates.
Communication Events & Attributes Event Over-Topping Level Reached Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI Communication Events & Attributes Event Gate Control Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% H Time Synch. Y
BER / PER …
Actor 2 Plant RTU Packet Loss Security 5% H Time Synch. Y
BER / PER …
All other attributes are the same as for the hydro plant general telemetry use case.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
123
Figure 56.
Hydro Plant Water Over-Topping Protection Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
124
4.3.8 Hydro Plant Turbine Vibration Monitoring Turbine vibration monitoring is done using a series of sensors in the turbines and other equipment to identify possible vibrations that might provide useful information for proactive maintenance and can also help identify and avoid harmful situations that cause damage to the turbine and associated systems.
Communication Events & Attributes Event Turbine Vibration Monitoring Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI
Figure 57.
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% H Time Synch. Y
BER / PER …
Hydro Plant Vibration Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
125
4.3.9 Hydro Plant Dam Deformation Monitoring Dam deformation monitoring is done to monitor the integrity of the dam structure and helps avoid possible issues with catastrophic consequences.
Communication Events & Attributes Event Dam Deformation Monitoring Mechanism Protocol Bandwidth Event61850-7-410 20 kbps triggered Redundancy Availability Direction Y H BI
Figure 58.
Actor 1 Plant RTU Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate …
Actor 2 Control Center Packet Loss Security 5% H Time Synch. Y
BER / PER …
Hydro Plant Dam Deformation Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
126
4.3.10 Wind Farm Operation There is about 1,349MW in Hydro Quebec’s generation portfolio from 15 wind farms operated by independent power producers. IEC standard 61400-25 defines standardized models for SCADA and data exchange and their corresponding mapping to communication profiles for monitoring and control of wind power plants.
Use Case General Description Title Wind Farm Operation Description Monitoring and control of wind farms Actors RTU; Control Center
Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 61400-25 & 20 kbps 60870-5-104 Redundancy Availability Direction
X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge
Trans. Rate
Date August 26, 2014
X X
Actor 2 Control Center Packet Loss Security 5% L Time Synch.
BER / PER
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
127
Y
M
BI
Communication Events & Attributes Event Event Mechanism Protocol Bandwidth Ad hoc 61400-25 & 20 kbps 60870-5-104 Redundancy Availability Direction Y M BI Communication Events & Attributes Event Control Mechanism Protocol Bandwidth Event61400-25 & 20 kbps triggered 60870-5-104 Redundancy Availability Direction Y H BI
Y
1 / 2 Seconds
Actor 1 RTU Latency 0.5 Sec
Jitter N/A
Acknowledge Y
Trans. Rate …
Actor 1 Control Center Latency Jitter 0.5 Sec N/A Acknowledge Y
Trans. Rate 1 / 2 Seconds
Date August 26, 2014
Y
…
Actor 2 Control Center Packet Loss Security 1% L Time Synch. Y
BER / PER …
Actor 2 RTU Packet Loss Security 1% M Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
128
Figure 59.
Wind Farm Operation Tabular & Graphic Information
In reality since H-Q does not operate these wind farms, the wind farm information is received by H-Q through an ICCP link between the wind farm operator’s control center and the H-Q Control Center. This scenario is shown in the following diagram.
Figure 60.
Wind Farm Operation at H-Q Graphic Information
For the ICCP link network requirements refer to the ICCP use case description in the Transmission use cases section.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
129
4.3.11 Wind Farm Maintenance The random behavior of wind speed determines the energy produced by a wind turbine. The same factor also influences the turbine degradation process due to the random load that the turbine is subjected to. As a result, wind turbines undergo a degradation process more complex than the equipment that work under stationary conditions. Wind farms are typically instrumented to detect early signs of failure and addressing the issue to avoid disruptions
and therefore increasing the system’s uptime. Wind turbine technological diversity and geographical spread of wind farms present significant operation and maintenance challenges. The solution is continuous remote monitoring of all operating parameters, both electrical and mechanical, including speed, frequencies of vibration on turbine components, oil pressure, gearbox assembly, etc. This data is processed at the local SCADA system within the farm and also transmitted to a centralized maintenance and monitoring center to be analyzed for preventive and corrective maintenance. Analysis of available telemetry data using condition-monitoring systems and algorithms and dispatching crew to address the issues before failure is critical to decreasing operating costs.
Use Case General Description Title Wind Farm Maintenance Description Continuous monitoring of the mechanical and electrical components in the wind farm for preventive and corrective maintenance. Actors Sensors/IED; Local SCADA; Data Center Business Service Category Transmission Distribution Energy Supply Demand Side
X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise
X X X
Places in the Communication Network Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
130
Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 61400-25 & 20 kbps 60870-5-104 Redundancy Availability Direction N M BI Communication Events & Attributes Event Event Mechanism Protocol Bandwidth Polled 61400-25 & 20 kbps 60870-5-104 Redundancy Availability Direction N M BI Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 61400-25 & 20 kbps 60870-5-104 Redundancy Availability Direction N M BI Communication Events & Attributes Event Event Mechanism Protocol Bandwidth Polled 61400-25 & 20 kbps 60870-5-104 Redundancy Availability Direction N M BI
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Sensors/IED Latency Jitter 1 Sec N/A Acknowledge
Trans. Rate …
Actor 1 Sensors/IED Latency Jitter 1 Sec N/A Acknowledge
Trans. Rate …
Actor 1 Local SCADA Latency Jitter 1 Sec N/A Acknowledge
Trans. Rate …
Actor 1 Sensors/IED Latency Jitter 1 Sec N/A Acknowledge
Trans. Rate …
Date August 26, 2014
X X
Actor 2 Local SCADA Packet Loss Security 5% L Time Synch. Y
BER / PER …
Actor 2 Local SCADA Packet Loss Security 5% L Time Synch. Y
BER / PER …
Actor 2 Maintenance Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
Actor 2 Maintenance Center Packet Loss Security 5% L Time Synch. Y
BER / PER …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
131
Figure 61.
Wind Farm Maintenance Tabular & Graphic Information
However, since H-Q does not maintain these wind farms, the wind farm information could be received by H-Q through an ICCP link between the wind farm operator control center and the H-Q Control Center, as shown below.
Figure 62.
Wind Farm Operation at H-Q Graphic Information
For the ICCP link network requirements refer to the ICCP description in the Transmission use cases section.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
132
4.4 Extreme Contingencies Based on a detailed analysis of data and accumulated experience from significant contingencies over the past 30+ years Hydro Québec has developed defensive plans against extreme contingencies to safeguard the reliability of the system for continuity of service to its customers.
4.4.1 H-Q Transmission System Characteristics Hydro Québec’s long transmission lines, harsh weather, and customers’ heavy reliance on electricity, altogether demand the highest level of security and reliability considerations in system design. H-Q’s transmission system is designed to have successive line of defense to counter events that are increasingly more severe but also increasingly more rare. H-Q’s system has no synchronous links with neighboring systems. Furthermore, the extensive 735kV transmission network has a relatively limited number of lines located in two major corridors, each about a 1000km in length. Because of these characteristics, stability and voltage control become critical issues. The first major corridor extends northwest up to the James Bay hydroelectric complex (15 000 MW) and the second corridor extends northeast up to Churchill-Falls and integrates about 14,000 MW of generation. There are also thirty-one 735kV substations, 11,200 MVAR in series compensation, one 1200-km 450-kV dc line, dynamic shunt compensation consisting of 11 static compensators and nine synchronous compensators, and about 3,900 MW of DC interconnections with neighboring systems. The following figure shows the layout of the 735kV transmission system4.
4
Designing a Reliable Power System: Hydro Québec’s Integrated Approach
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
133
Figure 63.
Hydro Québec’s 735kV Transmission System
4.4.2 Extreme Contingencies Criteria Disturbances that can affect the electrical and mechanical integrity of the system fall into three categories: 1. Natural causes (lightening, storms, cold, ice, forest fires, and geomagnetic storms) 2. Equipment outage or protection system failures 3. The human factor (operating errors, vandalism, design flaws, etc.) Depending on the triggering disturbance event, H-Q’s system is exposed to a diverse set of power system Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
134
phenomena: • • • •
Transient instability Dynamic instability (interregional oscillations at 0.5 Hz) Voltage instability Frequency instability (over- or under-frequency)
Hydro Québec’s concept of successive line of defense hinges on 4 principles: Principle 1: Service continuity must be assured following events most likely to occur on the power system. Principle 2: H-Q’s power system must include ways of avoiding system-wide power failures under extreme contingencies. Principle 3: Strategic equipment must not sustain any damage in the event of a general outage to ensure that system restoration is always an option. Principle 4: H-Q’s transmission system must be designed so as to allow the system to be restored within a reasonable period after a catastrophic event. The following figure summarizes H-Q’s concept of successive line of defense.
Figure 64.
5
Hydro Québec’s Concept of Successive Line of Defense 5
Designing a Reliable Power System: Hydro Québec’s Integrated Approach
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
135
The following figure shows the event(s) that constitute an extreme contingency in H-Q’s transmission system and the performance that is required in each case.
Figure 65.
Contingencies & Corresponding Performance Requirements 6
4.4.3 Special Protection Schemes (SPS) Hydro Québec’s has devised the following automatic actions and special protection schemes to counter the extreme contingencies.
6
Designing a Reliable Power System: Hydro Québec’s Integrated Approach
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
136
Special Protection Schemes
7
10
Level 1 Limited Action Level 2 Use of Corrective SPS Types
Level 3 Use of SPS with massive actions
Loss of two series or parallel 735kV lines AC-DC event; loss of a bipolar line with loss of one 735kV line Loss of a generating station or generation unit at a station Sudden loss of a major load center Unintended operation of an SPS Loss of lines and bypass of all series capacitors on the remaining lines in the same corridor Loss of all 735kV lines in a corridor Loss of all 735kV lines originating from a substation
Figure 66.
UnderFreq. Load Shedding (UFLS) 9
Generation Rejection (GR) and 12 Remote Load Shedding (RLS)
UnderVoltage Load Shedding 8 (UVLS)
Limited Generation Rejection (GR)
Shunt Reactor Tripping
Extreme Contingencies
Shunt Reactor Closing
Response Levels
Remote Tripping of Shunt Reactor 13 (RTS)
RPTC
11
MAIS
X
X
X
X
X
X
X
X
X
X
X X
X X
X
X
X
X
X
X
X
X
X
X
Extreme Contingencies & Relevant Special Protection Schemes
As can be seen from the above figure, H-Q’s Special Protection Systems include: •
Automatic Shunt Reactor Systems (MAIS) – installed in twenty-two 735kV substations that control about
7
MAIS – Automatic 735kV shunt reactor closing or tripping UVLS – Under-Voltage Load Shedding 9 UFLS – Under-Frequency Load Shedding 10 RPTC – Combination of GR, RLS, and RTS 11 GR – Generation Reaction 12 RLS – Remote Load Shedding 13 RTS – Remote tripping of shunt reactor 8
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
137
15,000 MVAR. Each MAIS device relies on a high precision, local voltage measurement while coordination between substations is done through switching-delay settings. •
Under-Frequency Load Shedding System (UFLS) – installed in about 150 distribution substations that can access over 13,000 MW of load. UFLS is intended to restore the generation/load balance after an extreme contingency. Unlike other more interconnected systems, due to absence of synchronous links between H-Q’s system and its neighbors’, UFLS can be relied on as a selective and effective measure to detect loss of generation, up to the amount of the system’s largest power station (5,300 MW).
•
Under-Voltage Load Shedding System (UVLS) or (TDST) – able to shed 1,500 MW of load mainly in 735kV substations in Montréal area. Load shedding is considered as a last resort and a corrective measure against voltage instability particularly when there is a severe initial drop in voltages that cannot be tolerated for a long time. TDST is a response-based SP, due to the nature of long-term voltage instability, and relies on voltage measurements taken in 5 substations in the Montréal area with high prevision MAIS transformers. TDST operates in a pre-defined load shedding arrangement with its control restricted to certain percentage of the load and the set of distribution breakers that can be opened is pre-defined.
•
Generation Rejection, Load Shedding, and Remote Reactor Tripping at 735kV (RPTC) – designed to detect multiple line losses or series-compensated capacitor bank tripping in 15 strategic 735kV substations. In addition to traditional under-frequency load shedding actions, H-Q has implemented an extensive generation rejection and remote load-shedding scheme to handle transient angular instability problems. RPTC is an event-based SPS due to the speed of angle instability phenomenon. RPTC is an effective measure against contingencies that cause loss of transmission capacity, without involving a loss of load or generation that lead to rapid instability. To be effective, the corrective measures should be taken very fast, for example generation rejections and remote load shedding must be executed no later than 0.25 s and 0.33 s respectively after the event is detected 14. In each substation in the RPTC, there are local programmable devices called contingency composition units (CCU) that analyze individual losses of lines and individual losses of series compensation banks to classify the contingencies by severity and flag those for which actions are required. GR systems and RLS systems receive their respective orders to operate directly from the CCUs in each substation.
4.4.4 Telecom Requirements for Extreme Contingencies Due to the fast reaction times and the critical importance of SPS to maintain power system electrical integrity, SPS have the most stringent communication requirements in terms of availability and low latency. These requirements are highlighted in the following figure.
Network Parameter
14
Network Requirement
Hydro Québec’s Defense Plan Against Extreme Contingencies, 1999
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
138
Latency Delay Symmetry Required Jitter Availability / Reliability VPN Topology Precise Timing Required Bit Error Rate Link / Node Failure Delay Performance Management Physical Path Redundancy
Figure 67.
4 to 10ms Yes 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event DR Request Mechanism Protocol Bandwidth Ad hoc DLMS 9.6 kbps Redundancy Availability Direction
X
X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 DR Headend Latency Jitter 1 Min N/A Acknowledge Trans. Rate
Date August 26, 2014
X X X
Actor 2 AMI Concentrator Packet Loss Security 5% H Time Synch. BER / PER Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
141
N
L
BI
Communication Events & Attributes Event DR Request Mechanism Protocol Bandwidth Ad hoc DLMS 9.6 kbps Redundancy Availability Direction N L BI
Figure 68.
Y
…
Actor 1 AMI Concentrator Latency Jitter 1 Min N/A Acknowledge Trans. Rate Y …
Y
…
Actor 2 Smart Meter Packet Loss Security 5% H Time Synch. BER / PER Y …
Hour-Ahead Load Optimization with DR Through AMI Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
142
4.5.2 Electric Vehicle Charging This use case describes support for residential consumers with an EV charging device and vehicle to grid (V2G) services. The two-way communication with the charging device enables more accurate forecasting of potential peak demands. The solution may be as simple as an additional EV Smart Meter or it can be extended to a sophisticated EV charging point whereby V2G services are transacted through a service provider. Once the charge / discharge status of the vehicle is known, the second step is to enable intelligent charging based on environmental and grid conditions. This may be driven by demand management signals such as Time of Use (TOU) tariffs or by direct load control signals. Use Case General Description Title Electric Vehicle Charging Description Electric vehicle charging and V2G services. Actors Actors Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify) X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Monitor Mechanism Protocol Bandwidth Polled 61850 64 kbps Redundancy Availability Direction N M BI
X
X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Charging Device Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Date August 26, 2014
X X X
Actor 2 Control Center Packet Loss Security 5% M Time Synch. BER / PER Y … Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
143
Communication Events & Attributes Event Event Mechanism Protocol Bandwidth Polled 61850 64 kbps Redundancy Availability Direction N M BI
Actor 1 Charging Device Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Control Center Packet Loss Security 5% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Charge Request / Authorization Mechanism Protocol Bandwidth Ad hoc Proprietary 64 kbps Redundancy Availability Direction N M BI
Actor 1 Charging Device Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Control Center Packet Loss Security 5% M Time Synch. BER / PER Y …
Figure 69.
Electric Vehicle Charging Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
144
4.5.3 Automated Demand Response with Water Heaters Demand Response (DR) is a mechanism for managing customer load in response to supply conditions, such as a change in price, or in response to grid conditions, such as generation/load imbalance leading to potential outages and grid reliability incidents. In the past, most DR was implemented manually through a fax or a phone call. The utility would notify the DR participants a day in advance for lowering their consumption. Today, with the establishment of communication links to smart loads, DR can be activated in much shorter time frames. This use case describes the control of water heaters for demand response. There are generally two flavors of DR: 1) Slow DR – where the DR signals are sent significantly before the events are called, e.g. day-ahead. Typical slow DR lead times are in hours and days. 2) Fast DR – where faster than usual response times are required. These programs have lead times of seconds (e.g. 4 sec response time), used for load balancing and frequency stabilization, for example ancillary services and regulation services. The current industry standard for automated demand, OpenADR216, is a flexible protocol to facilitate information exchange between energy service providers, aggregators, and end users. It enables the implementation of two-way signaling between service providers and automated clients, or in OpenADR terminology the Virtual Top Node (VTN) and Virtual End Node (VEN). The following figure 17 shows the possible relationships of VTN and VEN in OpenADR.
16
OpenADR Alliance is an industry group for the development, adoption, and compliance of the Open Automated Demand Response (OpenADR) standards. 17 OpenADR 2.0 Profile Specification – B Profile Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
145
Figure 70.
Possible Relationships of VTN and VEN in OpenADR2 [Source: OpenADR2 Spec]
Operational Security Assumptions & Principles With regard to securing demand side networks and resources, there are useful assumptions and principles offered in the “OpenADR Security Profile”18 document that we would like to summarize here: • • • • • •
•
18
Risks associated with the compromise of each DR asset will be different based on the compromised DR asset’s capabilities. DR is not part of critical grid operations unless DR resource/asset gives full commitment to accurately follow DR instructions. DR resource can be used to enhance grid reliability or to facilitate market operations. However, regulations and legal agreement require a separation between electric system operations and market functions. DR controlling entity (e.g. the ISO/RTO, Transmission Operator, DR Aggregator), may have little or no control over the physical environment in which DR assets reside in. Security controls should have minimal impact on the primary mission of DR. OpenADR should employ different types of security measures depending on the risks associated with different types of DR events in order to facilitate efficient operations of OpenADR applications: o If personally identifiable information is communicated, confidentiality becomes important. o If direct load control is introduced, integrity becomes important. o If faster response times are required, availability and low latency become important. All control activity on the DR system shall be auditable.
“Security Profile for OpenADR”, 2011
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
146
•
The integration of DR systems should not expose other utility systems to unauthorized access or attack. DR systems should support non-repudiation of all transactions between the DR controlling entity and DR resource/asset.
In this use case we describe the “B Profile” of OpenADR2. OpenADR2 messages are sent over the Internet using HTTPS protocol. Use Case General Description Title Demand Response using OpenADR2 Description Demand Response is a mechanism for managing customer load in response to supply conditions, such as a change in price, or in response to grid conditions, such as generation/load imbalance leading to potential outages and grid reliability incidents Actors DR Service Provider; DR Participant Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
Service Provider
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X
Communication Events & Attributes Event DR Event Mechanism Protocol Bandwidth Push or Pull HTTP 20 kbps Redundancy Availability Direction N L BI
X
X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 DR Service Provider Latency Jitter 1 Min N/A Acknowledge Trans. Rate Y …
Date August 26, 2014
X X
Actor 2 DR Participant Packet Loss Security 5% M Time Synch. BER / PER N …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
147
Communication Events & Attributes Event Report Mechanism Protocol Bandwidth Push or Pull HTTP 64 kbps Redundancy Availability Direction N L BI
Actor 1 DR Service Provider Latency Jitter 1 Min N/A Acknowledge Trans. Rate Y …
Actor 2 DR Participant Packet Loss Security 5% M Time Synch. BER / PER N …
Communication Events & Attributes Event Register Mechanism Protocol Bandwidth Push or Pull HTTP 64 kbps Redundancy Availability Direction N L BI
Actor 1 DR Service Provider Latency Jitter 1 Min N/A Acknowledge Trans. Rate Y …
Actor 2 DR Participant Packet Loss Security 5% M Time Synch. BER / PER N …
Communication Events & Attributes Event Opt In / Out Mechanism Protocol Bandwidth Push or Pull HTTP 20 kbps Redundancy Availability Direction N L BI
Actor 1 DR Participant Latency Jitter 1 Min N/A Acknowledge Trans. Rate Y …
Actor 2 DR Service Provider Packet Loss Security 5% M Time Synch. BER / PER N …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
148
Figure 71.
Demand Response Using OpenADR2 Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
149
4.5.4 Customer Premise Network Integration This use case describes integration between the utility distribution system and the customer premise network to give the utility a better understanding of actions behind the meter, e.g. distributed generation profile, and varying load a profiles that affect distribution operations. For customers’ participation in grid related transactions, be it market-driven or initiated by a utility call for reliability events is best handled through existing protocols such as OpenADR, whereby message exchanges between all participating parties are pre-defined and roles and responsibilities of each party is spelled out. For interactions that take place between the utility and a group of customers that are within the same local area (a neighborhood or a community) there are two possible scenarios: 1. The customers are represented through an aggregator or an energy service provider whereby the aggregator acts as a proxy for customers in all interactions with the utility. There are two contracts, one between the aggregator and the utility, and another between the aggregator and the customers. This applies for example to customers who have a leased rooftop PV system installed and operated by a service provider that guarantees the customer protection against rate changes and rate increases. 2. The second scenario is when customers are part of a microgrid. Their overall energy usage is balanced by existing active resources (distributed generation and storage) within the neighborhood or campus) and any excess power (or shortage of power) is balanced by the grid. In this scenario once can think of the possibility of an isolated microgrid whereby the system is separated from the grid and is operated autonomously. The system can be reconnected back to the grid in a controlled process that synchronizes frequency and angle at the point of common coupling. To this date there are not many., if any, systems whereby the utility has direct control over smart appliances inside the residential premise, except for those that are already covered under some type of demand response agreement that we address in the previous section. Typically, control of in-house or on-premise devices are entirely up to the site owner with little utility involvement. The only visible interface is either through the meter (as discussed under AMI use case) or through some form of demand response as we have seen previously.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
150
4.6 Mobility & Collaboration This section includes mobility and collaboration use cases including voice, video, and data connectivity for the mobile workforce, radio interoperability, and field coordination for emergency response and service restoration.
4.6.1 Field Workforce Voice over IP (VoIP) When the mobile worker at the substation makes a phone call, a session is established to the destination phone, via the IP PBX. Once the call session is established, VoIP packets flow directly between the IP phones.
Use Case General Description Title Field Workforce – IP Telephony Description Phone call from a substation to outside using IP Telephony. Actors Field Handset; Desktop Handset; Call Manager Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Voice Stream Mechanism Protocol Bandwidth Stream RTP 80 kbps Redundancy Availability Direction
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Field Handset Latency Jitter 150ms 30ms Acknowledge Trans. Rate
Date August 26, 2014
X X
Actor 2 Desktop Handset Packet Loss Security 1% L Time Synch. BER / PER Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
151
Y
H
BI
Communication Events & Attributes Event Call Control Mechanism Protocol Bandwidth On Demand RTP 150 kbps Redundancy Availability Direction Y H BI
Figure 72.
Y
…
Actor 1 Call Manager Latency Jitter 0.5 Sec N/A Acknowledge Trans. Rate Y …
N
…
Actor 2 Field Handset Packet Loss Security 1% L Time Synch. BER / PER N …
Field Worforce Voice over IP Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
152
4.6.2 Workforce Video Real-time collaboration through PC’s, laptops, tablets, or smart mobile devices is a unified communications application that is designed to make end users more productive by enabling them to communicate and collaborate easily with others in remote locations using a range of different communication options, including instant messaging (IM), voice, desktop sharing and video.
Use Case General Description Title Field Workforce Video Description On-demand video connection between two locations over IP network Actors Video Camera; Video Viewer; Video Manager Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Video Stream Mechanism Protocol Bandwidth Stream Video RTP 384 kbps Redundancy Availability Direction Y H BI
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Field Camera Latency Jitter 150ms 30ms Acknowledge Trans. Rate Y …
Date August 26, 2014
X X
Actor 2 Desktop Video System Packet Loss Security 1% L Time Synch. BER / PER N …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
153
Communication Events & Attributes Event Video Control Mechanism Protocol Bandwidth Polled H.232 150 kbps Redundancy Availability Direction Y H BI
Figure 73.
Actor 1 Video Manager Latency Jitter 1 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Desktop Video System Packet Loss Security 1% L Time Synch. BER / PER N …
Field Workforce Video Over IP Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
154
4.6.3 Radio over IP (RoIP) This use case describes the transfer of land mobile radio (LMR) audio signals over the IP network. A Land Mobile Radio (LMR) system is a collection of portable and stationary radio units designed to communicate with each other over predefined frequencies. They are widely deployed by utilities because of the need to have instant communication between geographically dispersed and mobile personnel. In typical LMR systems, a central dispatch console or base station controls communications to the disparate handheld or mobile units in the field. Within an organization, the radio systems tend to be homogenous, with most elements typically purchased from the same manufacturer. Although the electromagnetic spectrum is rather vendor agnostic, signaling mechanisms and other control aspects of individual radio systems can be quite proprietary. And if the utility needs to consolidate operations that were previously using different LMR systems, issues with interoperability could require workarounds to bridge the existing systems or ultimately require the purchase of all new equipment. With the LMR over IP service, standards-based VoIP technology voice gateways are used in combination with additional LMR specific features to address interoperability, extending command and control, and other issues. Base stations, repeaters, and dispatch consoles generally possess a wired interface that can be used to monitor audio received from their air interface, and as input for audio to be transmitted on their air interface. Although this wired interface may contain other control capabilities as well, as long as it has some sort of speaker output and microphone input, it can be connected to a voice port on a router. The audio received on the voice port is encoded with a standard audio codec, such as G.711 or G.729. Those audio samples are packaged in standards-based Real-Time Transport Protocol (RTP) packets suitable for transport on an IP network. At this point, the communication element is abstracted from the distinctive characteristics of each radio system, thus providing a solution for the interoperability problem. Now, these audio packets can be sent across the network to other LMR gateways with different brands of radio systems either individually (unicast) or as a group (multicast). The recipient of the audio packets need not be another LMR gateway. It can be any device capable of receiving and decoding the RTP stream, such as an IP telephone or PC with appropriate software. The IP network and IPenabled devices can be used to allow users to monitor or transmit on a particular radio channel from a desk without issuing another radio. This can be done locally, nationally, or internationally, assuming the IP network has been properly designed.
Use Case General Description Title Field Workforce – Land Mobile Radio (LMR) over IP (RoIP) Description Connecting disparate land mobile radio systems over the IP network and patching radio systems over terrain and geographies with no radio coverage. Actors Radio Repeater Router; RoIP Server; Radio Dispatch Console
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
155
Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
X X
Communication Events & Attributes Event RoIP Stream (Voice + Control) Mechanism Protocol Bandwidth Stream RTP 200 kbps Redundancy Availability Direction Y H BI
Actor 1 Radio Repeater / Router Latency Jitter 150ms 30ms Acknowledge Trans. Rate Y …
Actor 2 RoIP Server Packet Loss Security 1% L Time Synch. BER / PER N …
Communication Events & Attributes Event RoIP Stream (Voice + Control) Mechanism Protocol Bandwidth Stream RTP 200 kbps Redundancy Availability Direction Y H BI
Actor 1 RoIP Server Latency Jitter 150ms 30ms Acknowledge Trans. Rate Y …
Actor 2 Radio Dispatch Console Packet Loss Security 1% L Time Synch. BER / PER N …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
156
Figure 74.
Radio over IP (RoIP) Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
157
4.6.4 Substation Worker Access to Corporate Applications This use case focuses on the network connectivity for substation technicians and engineers within the substation. Currently, technicians / engineers perform work using information that is either from paper documents or from “ruggedized” laptop computers brought to the field. These computers contain large amounts of useful information (e.g. asset information, maps, work orders, and manuals). However, using a standalone computer has the following limitations: •
Information is not always current because it is not updated throughout the day in real-time. The field worker must connect with the network at the office to obtain updated information.
•
Back office systems do not obtain updated information from the field (for example, status of current work orders, missing or incorrect asset information, inspection results, and work orders generated in the field) until the work is completed and the field worker returns to the office to turn in completed paperwork or connects their laptops to the network.
This lack of automated information exchange between field workers and the back-office leads to operating inefficiencies (errors, delays and reduced work capacity), as well as outdated asset information. Providing secure IP network access to the devices in the substation and to the office enables: • • • • •
Technician / engineer collects information in the field and perform asset inspection, as well as planned and unplanned work assignments. The technician/power engineer may be using a hardened PC or a “ruggedized” laptop. Technician / engineer in substation accesses real-time information from the intranet (utility corporate network) including the utility’s asset management system. Technician / engineer in substation can access information from the intranet (utility corporate network) regarding assets: data sheets, manuals and guides, and troubleshooting instructions based on the initial analysis of the situation. Technician / engineer in substation updates information in the asset management system based on the current status of the equipment. This allows engineers/technicians to add the required details as they work on the case, providing accurate and complete information. Technician/engineer in substation accesses information from the Internet for third-party equipment information.
This use case should account for both wired and wireless access within the substation. The user needs to be authenticated and depending on the corporate application that is accesses the data may need to be encrypted. Network management is important for assigning IP addresses and managing ruggedized PCs, access points, and laptops. Within the Operational network these collaboration applications should be segregated from the SCADA network traffic all the way to the enterprise head-end and the corporate network should be separated from the SCADA network via centralized back-to-back firewalls.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
158
Use Case General Description Title Worker access to corporate applications from a substation. Description The technician/engineer can access data within the substation as well as securely access the application on the corporate network. Actors Field Worker PC/Laptop; Corporate Application Business Service Category Transmission Distribution Energy Supply Demand Side
X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Corporate Traffic Mechanism Ad hoc Redundancy N
Protocol TCP/UDP Availability M
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 PC
Bandwidth 64 kbps Direction BI
X X X X X
Latency 1 Sec Acknowledge Y
Jitter N/A Trans. Rate …
Date August 26, 2014
X X
Actor 2 Corporate Application (HTTPs, Citrix, etc.) Packet Loss Security 5% M Time Synch. BER / PER Y …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
159
Figure 75.
Substation Worker Access to Corporate Applications Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
160
4.7 Physical Safety & Security This section includes use cases related to physical security of the infrastructure and safety of the workforce and the public. Physical security use cases are motivated by the necessity to protect the grid infrastructure from internal and external physical threats and by the need for compliance with regulatory requirements such as North American Electric Reliability Corporation’s Critical Infrastructure Protection (NERC/CIP) standard CIP006. For example, version 5 of CIP006 (the latest version at the time of this writing) calls for mandatory security measures to protect those grid assets that if compromised will impact the reliability and stability of the grid. Specific CIP006 mandates include: • • •
Establishment of a physical security program Physical access controls for high- and medium-impact grid assets Monitoring physical access to important grid facilities such as control centers, high voltage substations, and any other with significant impact on grid reliability • Utilizing 2 or more physical access controls to collectively restrict access • Logging physical access to grid facilities • Retaining access logs • Maintenance and testing of physical security measures to provide compliance with the regulatory requirements For physical safety and security we study the following use cases: 1. 2. 3. 4. 5.
Electronic access control Video surveillance Fire alarms Substation environmental monitoring A composite use case for worker safety in the case of a fault
4.7.1 Electronic Access Control This service provides electronic monitoring and control for physical access to remote areas, building and locations either by using badges, card readers, and keypads or remotely from a Control Center over the communications network.
Use Case General Description Title Electronic Access Control Description Using electronic sensors and actuators, e.g. door sensors and badge readers and remote control from a security control center to authorize access to a restricted area. Actors Physical Access Gateway; Security Control Center
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
161
Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
X X
Communication Events & Attributes Event Door Position Mechanism Protocol Bandwidth Event TCP based 5 kbps Redundancy Availability Direction N M BI
Actor 1 Physical Access Gateway Latency Jitter 1 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Security Control Center Packet Loss Security 5% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Entry Request Mechanism Protocol Bandwidth Event TCP based 5 kbps Redundancy Availability Direction N M BI
Actor 1 Physical Access Gateway Latency Jitter 1 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Security Control Center Packet Loss Security 1% M Time Synch. BER / PER Y …
Communication Events & Attributes Event Entry Request Mechanism Protocol Bandwidth Event TCP based 5 kbps Redundancy Availability Direction N M BI
Actor 1 Physical Access Gateway Latency Jitter 1 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 Security Control Center Packet Loss Security 1% H Time Synch. BER / PER Y …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
162
Figure 76.
Electronic Access Control Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
163
4.7.2 Video Monitoring & Surveillance Video surveillance provides the control center operators the capability of viewing live events as well as the retrieval and analysis of past security events by accessing the Digital Video Recorders (DVR). The benefits of live video surveillance are: • • • •
Detection of an actual or attempted security breach at a substation Providing intelligence for security monitoring of critical assets Cost savings in responding to alarms Cost savings associated with security monitoring and random patrol fees.
Furthermore, • •
Infrared cameras could be used for thermal monitoring of transformers, and Pan-Tilt-Zoom (PTZ) cameras can be used to visually monitor breakers.
The same pattern in deployment of sensors for remote monitoring applies to other monitoring such as: • • •
Motion detection Gas leak detection Earthquake detection
Typically there is a policy-based minimum retention period for video files, e.g. 3-30 days. File storage can be done locally at the premise. After the initial retention period files are moved to off site storage.
Use Case General Description Title Video Monitoring & Surveillance Description Video surveillance for remote monitoring of restricted premises; cost savings in truck rolls and personnel time, and asset condition monitoring. Actors Video cameras and Digital Video Recorder (DVR); Security Control Center Business Service Category Transmission Distribution Energy Supply Demand Side
X X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Date August 26, 2014
X X
X X X X Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
164
Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X X
Communication Events & Attributes Event Video Control Mechanism Protocol Bandwidth On demand RTP 256 kbps Redundancy Availability Direction N M BI
Communication Events & Attributes Event Video Stream Mechanism Protocol Bandwidth Stream RTP 256 kbps Redundancy Availability Direction N M BI
X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
X X
Actor 1 Security Control Center Latency Jitter 1 Sec N/A Acknowledge Trans. Rate Y …
Actor 2 DVR Packet Loss Security 1% M Time Synch. BER / PER Y …
Actor 1 Camera
Actor 2 Security Control Center Packet Loss Security 1% M Time Synch. BER / PER Y …
Latency 1 Sec Acknowledge Y
Jitter N/A Trans. Rate …
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
165
Figure 77.
Video Monitoring & Surveillance Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
166
4.7.3 Remote Fire Alarms Monitoring Remote monitoring of fire alarms enables the Control Center personnel to quickly locate and react to possible fires in remote locations using smoke detectors and fire sensors. The existing fire alarm panels ('FAP') within H-Q’s footprint have no IP connectivity, and alerts are communicated via SCADA to the electrical control room. However, in circumstances where a FAP is experiencing a fault, there is currently no way to diagnose the issue remotely. Instead, the root cause of the issue can only be identified when the FAP maintainer reaches the site and is able to connect to the panel via a laptop. While this can lead to significant delay, especially in rural sites, this also leads to multiple site visits, as the maintainer may not arrive at site with the parts needed to resolve the issue. Fire Services Officers and safety staff also have no ability to remotely audit FAPs to ensure parts of the system have not been left isolated.
Use Case General Description Title Remote Fire Alarms Monitoring Description Remote monitoring of fire alarms for early detection and timely response without having to dispatch personnel to the site. Actors Actors Business Service Category Transmission Distribution Energy Supply Demand Side
Mobility & Collaboration Physical Safety & Security Other (Specify)
X
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission) Communication Events & Attributes Event Alarm
X X X X
X X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 Alarm System
Date August 26, 2014
X X
Actor 2 Security Control Center Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
167
Mechanism Eventtriggered Redundancy N
Protocol TCP based
Bandwidth 5 kbps
Latency 1 Sec
Jitter N/A
Packet Loss 1%
Security M
Availability M
Direction UNI
Acknowledge Y
Trans. Rate …
Time Synch. Y
BER / PER …
Figure 78.
Remote Fire Alarms Monitoring Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
168
4.8 Other Use Cases This section is added for completeness and includes use cases that don’t fit within the other categories.
4.8.1 Generic Network Management Use Case This use case describes the monitoring of the communication network, including device status, device configuration, latency test, software management, and threshold testing by the Network Management System. The benefits include improved visibility of communication network faults; reduced communications minutes lost due to undetected faults, and proactive maintenance of communications assets.
Use Case General Description Title Communication System Monitoring Description Monitoring of the communication system Actors Network Management System; Communication Device (Router/Switch) Business Service Category Transmission Distribution Energy Supply Demand Side
X X
Mobility & Collaboration Physical Safety & Security Other (Specify)
Places in the Grid Data Center / Control Center Centralized Generation Plant / Storage Facility - with capacity > 100MVA Distributed Generation / Storage Resource with capacity < 100MVA High Voltage Transmission Substation with voltage level > 161KV Medium Voltage Sub-Transmission Substation with voltage level 69KV to 161KV Low Voltage Distribution Substation with voltage level - 25KV Distribution Line / feeder at medium or low voltage levels Customer Premise Places in the Communication Network Control Center Data Center Wide Area Network (WAN) Substation Network (Transmission)
X X X
Communication Events & Attributes Event Configuration Mechanism Protocol Bandwidth Ad hoc SSH 64 kbps
X
X X X X
Generation / Storage Plant Network Field Area Network (Distribution) Commercial / Industrial Area Network Residential Premise Network
Actor 1 NMS Latency 1sec
Date August 26, 2014
Jitter N/A
Actor 2 Device Packet Loss Security 1% M Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
169
Redundancy N
Availability H
Direction BI
Communication Events & Attributes Event Threshold Event Mechanism Protocol Bandwidth Ad hoc SNMPv3 9.6 kbps Redundancy Availability Direction N H BI Communication Events & Attributes Event Device Status Mechanism Protocol Bandwidth Ad hoc SNMPv3 9.6 kbps Redundancy Availability Direction N H BI
Communication Events & Attributes Event Software Management Mechanism Protocol Bandwidth Ad hoc SCP 64 kbps Redundancy Availability Direction N H BI Communication Events & Attributes Event Latency Test Mechanism Protocol Bandwidth Ad hoc ICMP 9.6 kbps Redundancy Availability Direction N H BI
Acknowledge N
Trans. Rate
Actor 1 Device Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Actor 1 Device Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Actor 1 NMS Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Actor 1 Device Latency 1sec Acknowledge N
Jitter N/A Trans. Rate
Date August 26, 2014
Time Synch. Y
BER / PER …
Actor 2 NMS Packet Loss Security 1% M Time Synch. BER / PER Y …
Actor 2 NMS Packet Loss Security 1% M Time Synch. BER / PER Y …
Actor 2 Device Packet Loss Security 1% M Time Synch. BER / PER Y …
Actor 2 NMS Packet Loss Security 1% M Time Synch. BER / PER Y …
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
170
Figure 79.
Generic Network Management Tabular & Graphic Information
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
171
4.8.2 Precision Time Protocol (PTP) Hydro Québec do not use GPS clocks in generation substations. One of the main reasons is that some of the generation plants are 30 to 50 meters deep under ground and the GPS signal can be weak and unreliable. Instead, atomic clocks are used. There are forty Rubidium clocks and four Cesium clocks. Clocks are synchronized amongst each other and the synchronized time signal is sent through SONET. Rubidium clocks provide clock to the SONET network and 1ms timestamps for IRIG-B. Eventually, H-Q plans to transition to the Precision Time Protocol (IEEE 1588), distributing the synchronization signal over the IP/MPLS network. The Precision Time Protocol (PTP) is defined in IEEE standard 1588. PTP is applicable to distributed systems consisting of one or more nodes, communicating over a network. Nodes are modeled as containing a real-time clock that may be used by applications within the node for various purposes such as generating time-stamps for data or ordering events managed by the node. The protocol provides a mechanism for synchronizing the clocks of participating nodes to a high degree of accuracy and precision. PTP operates based on the following assumptions 19: • • • •
It is assumed that the network eliminates cyclic forwarding of PTP messages within each communication path (e.g., by using a spanning tree protocol). PTP eliminates cyclic forwarding of PTP messages between communication paths. PTP is tolerant of an occasional missed message, duplicated message, or message that arrived out of order. However, PTP assumes that such impairments are relatively rare. PTP was designed assuming a multicast communication model. PTP also supports a unicast communication model as long as the behavior of the protocol is preserved. Like all message-based time transfer protocols, PTP time accuracy is degraded by asymmetry in the paths taken by event messages. Asymmetry is not detectable by PTP, however, if known, PTP corrects for asymmetry.
A time-stamp event is generated at the time of transmission and reception of any event message. The timestamp event occurs when the message’s timestamp point crosses the boundary between the node and the network.
19
IEEE 1588, Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems Date August 26, 2014 Use Cases Proposal for IEC 172 © 2014 Cisco Systems, Inc.
5. Network Requirements Heat Map & Dependency This section presents two views of the communication requirements: 1. A matrix showing color-coded representation of use case telecom requirements. The color convention will follow the same rules that we defined for the UML diagrams. The matrix can be seen as a heat map showing where there are stringent requirements for telecom services. We have segmented the matrix by domain. 2. A dependency matrix indicating all places in the network that data packets for each use case traverse viewed from the network’s perspective, one can see all use cases that should be taken into account for the architecture and design of the network.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
173
5.1 Network Requirements Heat Map – By Use Case This section represents the main network requirements, i.e., latency, bandwidth, security, and availability, for all use cases. The representation is in matrix format where the cells are color-coded to indicate use cases that impose more stringent requirements on the communication network. Red indicates the most stringent requirement; yellow and green indicate relatively less stringent requirements, respectively. The matrix offers a visual heat map of the network requirements.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
174
Figure 80. Date August 26, 2014
Network Requirements Heat Map – by Use Case Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
175
5.2 Network Dependency Matrix – By Use Case The following matrix shows dependencies on different places in the network, for each use case.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
176
Figure 81. Date August 26, 2014
Network Dependency Matrix – by Use Case Use Cases Proposal for IEC
© 2014 Cisco Systems, Inc.
177
6. Communication Trends & Best Practices These general communication requirements are over and above the specific requirements of the use cases that have been addressed so far. These include both current and future communication related requirements that should be factored into the network architecture and design.
6.1 General communication Requirements IP Connectivity everywhere Establish IP connectivity everywhere. Features of the IP network must meet the needs of all services in terms of: • • • •
Security – cyber security and data integrity Quality – bandwidth and latency Reliability – redundancy Availability – reduced failure rates of devices, for example by minimizing the number of devices and by ruggedizing them as necessary
Monitoring services everywhere and from different remote centers Due to the big large and diversity of devices and manufacturers, it is necessary to be able to remotely connect to all of them in order to retrieve the product configuration and event log. Multisite capability is important in this case and is one of the reasons for recommending using IP protocol everywhere. Move services to a virtual data center Due to the legacy application some of the communication management application do not reside in the virtual data center. Some services are currently provided by servers outside the Virtual data centers and are therefore out of the security framework and maintenance plans. All application should be move and any new application must be VDC enabled. Unify access to applications / information from the corporate network This requirement is also related to moving all applications to the virtual Data center and ensuring that applications can be access from anywhere in the network (corporate or operational network infrastructures) with the appropriate level of security. Unify services Typically, it is common practice to install new infrastructure/solution when a new service is required instead of integrating the new one in the already installed infrastructure. This is especially evident while deploying remote services. This has lead to building a siloed network, which increases the complexity, is Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
178
less efficient than a converged network and increases management costs. The introduction of new services as well as the integration of existing ones is significantly easier using a unified or converged infrastructure and thus enables a unified service approach. Unified Communications Solutions A unified voice/video communication solution should provide connectivity to all remote sites, substations, and generation plants. Ensuring connectivity to many of the remote sites where vehicle access during the winter is difficult. Building a robust infrastructure for these locations is critical. Unified Fiber Technology – SONET replacement H-Q enhanced over time their SONET network to support DWDM and SONET-NG technologies. SONET is reaching its end of life and can’t compete with new technologies like IP over dense wavelength-division multiplexing (IPoDWDM) or IP/MPLS over dense wavelength-division multiplexing (IPoMPLSoDWDM), which provide packet switching instead of circuits-based networks. Standardize grid communication protocol to opened standard. Multiple protocols exist for substation automation, many of them being proprietary protocols with custom communication links. Such environment makes cross-vendor interoperability a challenge and increase additional OPEX and CAPEX to support multiple gateways and protocol translators. Standardizing the communication model is key to benefit from the full potential of an IP end-2-end network. IEC 61850 is the new standard communication model developed since the last 20 years and supported today by most of the vendors. The goals of the working group defining this protocol are • • • • •
A single protocol for complete substation considering modeling of different data required for substation. Definition of basic services required for transferring data so that the entire mapping to communication protocol can be made future proof. Promotion of high interoperability between systems from different vendors. A common method/format for storing complete data. Define complete testing required for the equipment that conforms to the standard.
This will enable the migration to an end-to-end IP infrastructure from sensor to control center. Reliable Communications for Distribution Substations The Fault Location Isolation & Service Restoration (FLISR) algorithm and its implementation demands reliable communications, at least with urban, and semi-urban where there is the possibility of feed from two different primary substations. Rest of the substations requires reliable communications to the extent that they can reliably receive and accept commands from the control center.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
179
IEEE 1588 time synchronization Client / Server Capabilities Use IEEE 1588 for time synchronization to enable a single clock source and remove the requirements to have GPS clocks in each location. This will enable an accurate centralized timing source to be implemented. The benefits are reduced costs and greater control over and lowering of some of the security threats that have been seen from using remote clock sources. The communication devices must be compatible with IEEE 1588 in order to be synchronized as a client. In addition, these devices should be able to behave as a server in order to provide the clock to the 1588 end-points. Accurate clock synchronization is required for the current infrastructure and long-term applications. The communication devices software and hardware must be IEEE 1588 complainant. Currently, Time is distributed using legacy, propriety protocols and using redundant GPS in some locations. H-Q generation plants are located deeply underground blocking any GPS signal. To provide time synchronization service, H-Q deployed the following architecture: - 4 Cesium atomic clocks, each one covering one area of the province and backing up each other. - Cesium clocks provide clock to 40 Rubidium clocks - Rubidium clocks provide clock to the SONET network and 1ms timestamps for IRB H-Q will use IEEE 1588 to provide time synchronization once all grid services will have been migrated to their IP/MPLS backbone. Integration of Multicast Design Integrate multicast into the high level design for point to multipoint traffic, such as PMU or video. Multicast technologies will become a requirement as we move to using real time data across multiple locations or when streaming data from devices such as PMUs. The ability for the communications network to effectively manage these data streams and to ensure that they get delivered to the correct destination in a timely fashion will become critical to the operation of the grid. Multicast is a key technology so the control center can reach thousands if not millions of devices using a single request. A basic use case is sensor firmware upgrade. New technology like Label Switch Multicast or LSM should be considered to transport multicast traffic over a MPLS backbone. It will support MVPN services without the need to enable PIM in the core and will benefit from feature like FRR. QoS Requirements Mapping Converging multiple types of applications into a shared multi-services infrastructure requires a strong QoS policy. Each application must be characterized from delay / jitter / latency / bandwidth perspective so class of services can be defined as well as the mapping policy of the application to those classes. Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
180
Enable Future Network Expansion The network design should allow for expansion of the network as future applications are added. It is hard to predict all of the services and applications that the network will be required to support over the next 15 years so it is vital that the designers build a flexible network architecture capable of expansion for both number of locations, number of devices, and the communications requirements such as bandwidth and latency. Substation Network Resilience Define criteria for categorizing substations based on their criticality and design the substation network resilience requirements based on the criticality of the sue cases that should be supported in each substation category. Fast Convergence Design Different applications have different requirements in terms of connectivity availability. Most critical application like Teleprotection needs the lowest convergence time. Fast convergence is not a feature you just need to enable. It’s an architecture that requires deep understanding of all the different routing protocols and links type used to provide end-to-end connectivity. The technical decision will be based on the failure scenarios that should be supported and the expected convergence time. Scalable Headend Design Extend the Smart Grid network design into the headend and data center. The communications network design must extend into the headed or control center architecture to ensure a full end-to-end design is achieved. This is critical when the application has low end-2-end latency budget and need the remote device needs to talk to the Control Center. Define Service Level Agreements (SLA) and Enable SLA Monitoring Within the high level architecture, build the network SLAs and define the methods of measuring them within the network management system. It is critical that the SLAs for the network are understood. This allows the design to not only ensure that they can be meet but also that we can use synthetic traffic monitoring controls by a network management system to monitor that they are being achieved. Integration of 3G/4G Technologies Build 3G/4G technology into the High Level Design. Review the use of WiMax and radio technologies within the architectural design. These may be used to connect to remote locations. Ethernet Connectivity for Station Bus Architecture Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
181
Build Ethernet architecture for Station bus connection and remove serial connections. This will simplify connectivity within the substation. This will also remove the requirement for multiple serial connections and removes the slow serial bus architectures that are used. This also ensures increased flexibility for integration into other systems in the substation such as physical security. Station bus resilience through standards such as PRP and HSR should be examined within the high-level design. IEC61850 Ethernet Station Bus will be required but this is a long-term goal. Ethernet Connectivity for Process Bus Architecture Enable Ethernet connectivity from RTUs to sensors and IEDs. This will simplify connectivity within the substation. This removes the requirement for multiple serial connections and removes the slow serial bus architectures that are used. This also ensures increased flexibility and increased speed with the use of multicast messaging between multiple devices. Ethernet Technologies will be deployed for the process bus using fiber optic technology. Protection and teleprotection on IP Integrate teleprotection into the IP network. Converging onto a single IP infrastructure offers considerable cost savings. However, the challenge is the integration of new components and systems such as remedial actions schemes and system integrity protection schemes that require integration of many systems that has not been integrated before. Running over an IP infrastructure using standard protocols such as IEC 61850 simplifies the deployment and management of the overall system. It also removes the tie between the physical communications infrastructure to the system logic. This will be increasingly important as we move to a distributed intelligence approach for grid monitoring and control.
6.2 Migration to Packet-Switched Network Throughout the world, utilities are increasingly planning for a future based on smart grid applications requiring advanced telecommunications systems. Many of these applications utilize packet connectivity for communicating information and control signals across the utility’s Wide Area Network (WAN), made possible by technologies such as multiprotocol label switching (MPLS). The data that traverses the utility WAN includes: • • • • •
Grid monitoring, control, and protection data Non-control grid data (e.g. asset data for condition-based monitoring) Physical safety and security data (e.g. voice and video) Remote worker access to corporate applications (voice, maps, schematics, etc.) Field area network backhaul for smart metering, and distribution grid management
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
182
•
Enterprise traffic (email, collaboration tools, business applications)
WANs support this wide variety of traffic to and from substations, the distribution grid, generators, between control centers, and between work locations and data centers. To maintain this rapidly expanding set of applications, many utilities are taking steps to evolve present time-division multiplexing (TDM)–based and frame relay infrastructures to packet systems. Packet-based networks are designed to provide greater functionality and higher levels of service for applications, while continuing to deliver reliability and deterministic (real-time) traffic support.
6.3 MPLS technology MPLS is a proven WAN technology for network operators who need to support diverse legacy systems as well as modernize for next-generation applications. Enabling transparent integration of traditional and smart grid capabilities, MPLS facilitates transport of most forms of traffic, from traditional serial-based technologies such as SCADA remote terminal units (RTUs) to today’s IEC 61850 packet-based intelligent electronic devices (IEDs). In the utility industry, MPLS is by far the most commonly selected WAN technology for smart grid implementations because of its: -
Maturity and proven capabilities across large-scale industrial and enterprise networks Ability to support both traditional applications and next-generation requirements Ability to virtualize the WAN into independent sub-networks Centralized management of physical infrastructure and virtualized sub-networks Ability to enhance and become an integral part of the security framework across the WAN Modularity for scalability and flexibility, as well the ability to protect the overall system from domain failures
By supporting multiple applications on a converged network, our smart grid solutions provide a framework for integrating new technologies and utility-specific applications. The modular approach enables implementation of projects over time, allowing utilities to plan their investments and flexibly adapt to changing business circumstances. MPLS offers a number of features that make it especially suitable for multiservice, high-security industrial environments. By protecting existing system investments while enabling the transition to the modernized grid, MPLS provides a secure, flexible, and high-performance foundation for utility systems.
6.3.1 Network Virtualization In high-security, geographically distributed environments, large flat networks are usually neither practical nor reliable. They create a single fault domain and do not permit the kind of modularity needed for dependable, secure system management over large distances. As a result, grid applications suffer due to congestion, poor performance, reliability and security issues, and administrative complexity. Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
183
Within an MPLS infrastructure, however, applications can be logically separated and secured to support specific business functions while remaining on a single physical network. These software-defined segments operate securely and independently from each other, minimizing the fault domain. Within each virtual network, MPLS supports deterministic traffic, prioritization of traffic flows, and flexible allocation of bandwidth to enable optimal, consistent performance. Traffic is secured by policy-based trust boundaries, with each operations system accessible only to an approved group of users. Administrators can also create new secure segments as needed based on scalable Layer 2 and Layer 3 virtual private networking (VPN) domains, including point-to-multipoint capabilities.
6.3.2 Support for Existing Networks & Interoperability One of the most valued features of MPLS is that it allows utilities to perpetuate the use of existing TDM circuits, ATM, frame relay, and other traditional communication networks on the same WAN backbone with nextgeneration packet-based systems. This is achieved either by running these legacy systems over an MPLS network using techniques such as circuit emulation with Pseudo Wire Emulation Edge-to-Edge (PWE3) and/or by overlaying MPLS onto an existing TDM-based network infrastructure. Enhanced by MPLS Traffic Engineering (TE), networks can integrate virtually all forms of traffic without having to disruptively replace still-functioning older systems. This helps to unify the network management environment, making it significantly more cost-effective to administer. By running new applications alongside older systems on the same network, utilities can protect their current investment while transitioning to a modernized grid.
6.3.3 Security in MPLS MPLS virtualization greatly enhances network security. By creating logical separation of routing and data flows, utilities are able to safeguard specific information for each segmented domain. In doing so, it assures greater security for a variety of applications. For example, many utilities are considering how they might improve mobile worker connectivity through Wi-Fi, especially in remote areas where public cellular signals are not always reliable. However, a security threat exists in the potential of Wi-Fi traffic mingling with operations traffic. This issue can be resolved by utilizing robust WiFi security mechanisms as well as MPLS to create a totally segregated virtual network for all Wi-Fi traffic. Along the way, MPLS mechanisms such as rate shaping, priority handling, and traffic engineering mechanisms help to guard against denial of service and other malicious attacks.
6.3.4 Utility-Grade Performance Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
184
Utilities have traditionally accepted SONET/SDH for its ability to deliver high-performance connectivity. By contrast, packet solutions have sometimes been characterized as “best-effort networks,” especially in situations where they are based on T1 or low-bandwidth connectivity. But this not true for well-designed packet networks, especially not for high-speed MPLS networks designed with Quality of Service, traffic engineering, fault detection, and Fast Reroute (FRR) features. In contrast to other forms of packet solutions, which function on a hop-by-hop basis, MPLS TE steers traffic across predetermined routes in case of a network failure. Features such as bidirectional forwarding detection (BFD) and FRR can detect failures and reroute traffic on a par with SONET, helping to assure reliability and speed of network traffic.
6.3.5 IP/MPLS and MPLS-TP for the WAN The flexibility of MPLS allows utilities to transport data using a variety of static and dynamic techniques, including fully switched or circuit-oriented connections, or quite commonly for a hybrid configuration supporting both. The appropriate deployment approach is determined by assessing such issues as the nature of the environment, the coverage area, the level of SONET/SDH functions, dynamic signaling, control plane policing as well as operational business requirements.
6.3.6 The Cost Efficiencies of MPLS A key benefit of the virtualized MPLS infrastructure is that it can help utilities to cut costs in a number of areas. It assists in capital management by eliminating duplicate equipment and minimizing spares and inventory. As well, asset management is improved with a less complex infrastructure and management capabilities. Utilities also avoid early depreciation with planned, timely investments in strategic equipment. Utilities have seen reduced operations and administrative costs of overlay networks, tools, and management systems by as much as 50 percent, according to industry studies. Based on the single network platform, organizations eliminate duplicate vendor support services and contracts, and optimize service provider contracts (and circuits). They minimize downtime with less maintenance and out of service conditions and reduce the potential for regulatory fines.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
185
6.4 IP Address Planning & Management 6.4.1 Overview IP addressing plan development and IP Addresses management are two fundamental activities for any IP based communication network. A good IP addressing plan implemented in a well-designed network provides scalability, predictability, and flexibility. IP address management is the process of allocating, recycling and documenting IP addresses and subnets in a network. IP addressing standards define subnet size, subnet assignment, network device assignments and dynamic address assignments within a subnet range. Recommended IP address management standards reduce the opportunity for overlapping or duplicate subnets, non-summarization in the network, duplicate IP address device assignments, wasted IP address space, and unnecessary complexity. The first step to successful IP address management is to understand the IP address blocks used in the network. Once the address blocks have been defined, allocate them to areas of the network in a way that promotes summarization. In many cases, these will have to be further subdivided based on the number and size of subnets within the defined range. Standard subnet sizes for standard applications should be defined, such as building subnet sizes, WAN link subnet sizes, loopback subnet size, or WAN site subnet size. Then these subnets can be allocated for new applications out of a subnet block within a larger summary block. The main focuses of a new IP addressing plan should be to: • • • • •
Provide additional control over network resource allocation Accommodate new applications requirements Support continuing growth Ease of route and network management Increase application, network, and system security
Utilities also have unique challenges from an IP addressing perspective: • • • •
Need to connect many (up to millions) field devices in a plug and play way (Zero touch deployment) Must take into account complete isolation between the station and the process buses within the substation In-line with NERC-CIP requirements Support new IP based applications like physical security, mobile workforce enablement, Precise Time distribution or IP telephony.
It is clear IPv4 can’t support all these requirements and Utilities must consider IPv6 when designing smart grid networks.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
186
6.4.2 IPv6 Considerations IPv6 integration is a critical factor to any smart grid network that is being designed today. IPv6 for smart metering is a must, as potentially millions of devices must be addressed. We also need to consider IPv6 within the main communications network, and while there are few substation devices that are IPv6 enabled today, the network is being designed to last 15 years and we do see a requirement for these devices to be deployed within that time frame. It’s recommended to secure a public IPv6 prefix sooner than later even if it’s not implemented right away. Utility of the size of H-Q should request a /36 prefix.
6.4.3 Critical Issues in IP Address Assignment Utilities typically have a repeatable plug and play approach when deploying RTUs or IEDs equipment in substation. These devices are all "pre-configured exactly the same way which greatly simplifies the maintenance and support model. When those same devices were IP-enabled, utilities continued to use the same practice resulting in multiple devices having the same IP address. This scheme relies heavily Network Address Translation (NAT) mechanism to allow communication with the control centers. This model works fine when with the control center(s). This model was adequate when the communication was mainly unidirectional and following a hub & spoke topology. However, this design is not recommended anymore, as it will not support new smart grid application requirements: •
Bidirectional Communication – If the application, hosted outside of the substation, needs to initiate a connection, complex Port Address Translation (PAT) rules must be deployed. Furthermore, it imposes a strong requirement at the application level, as each of them must use a unique application port.
•
Multi-point to Multi-point Communication – A substation may have multiple exit points to reach many different sites across the utility network. If NAT is required, it must be enabled at every single edge interface which makes the network much more difficult to manage and operate. Each time a new application is deployed, it may require update of the NAT rules for every NAT gateway deployed across the network.
•
Bottleneck – Most of the time NAT implementation is not stateless. It means the traffic in both directions must go through the same NAT gateway leading to a Single Point of Failure and potential performance bottleneck issues.
•
Application Support – Some applications are not NAT-compatible as they carry some IP information in their payload. To support such applications, an Application Level Gateway (ALG) must be embedded into the NAT gateway itself to apply the required translation rules to the payload of the packet.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
187
7. Security Trends & Best Practices 7.1 Current Practices & Their Limitations Grid monitoring and control devices are already targets for cyber attacks and legacy communication protocols have many intrinsic network related vulnerabilities. DNP3, Modbus, PROFIBUS/PROFINET, and other protocols are designed around a common paradigm of request and respond. Each protocol is designed for a “master” device such as an HMI system to send commands to subordinate “slave” devices to retrieve data (reading inputs) or control (writing to outputs). Because many of these protocols lack authentication, encryption, or other basic security measures, they are prone to networkbased attacks, allowing a malicious actor or attacker to utilize the “request and respond” system as a mechanism for “command and control” like functionality. Specific security concerns common to most industrial control, including utility communication protocols include the following: •
Network or transport errors (e.g. malformed packets or excessive latency) can cause protocol failure.
•
Protocol commands may be available that are capable of forcing slave devices into inoperable states, including powering-off devices, forcing them into a “listen only” state, disabling alarming.
•
Protocol commands may be available that are capable of restarting communications and otherwise interrupting processes.
•
Protocol commands may be available that are capable of clearing, erasing, or resetting diagnostic information such as counters and diagnostic registers.
•
Protocol commands may be available that are capable of requesting sensitive information about the controllers, their configurations, or other need-to-know information.
•
Most protocols are application layer protocols transported over TCP; therefore it is easy to transport commands over non-standard ports or inject commands into authorized traffic flows.
•
Protocol commands may be available that are capable of broadcasting messages to many devices at once (i.e. a potential DoS).
•
Protocol commands may be available to query the device network to obtain defined points and their values (i.e. a configuration scan).
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
188
• •
Protocol commands may be available that will list all available function codes (i.e. a function scan). Bump in the wire (BITW) solutions – A hardware device is added to provide IPSec services between two routers that are not capable of IPSec functions. This special IPsec device will intercept then intercept outgoing datagrams, add IPSec protection to them, and strip it off incoming datagrams. BITW can all IPSec to legacy hosts and can retrofit non-IPSec routers to provide security benefits. The disadvantages are complexity and cost.
These inherent vulnerabilities, along with increasing connectivity between IT an OT networks, make networkbased attacks very feasible. Simple injection of malicious protocol commands provides control over the target process. Altering legitimate protocol traffic can also alter information about a process and disrupt the legitimate controls that are in place over that process. A man- in-the-middle attack could provide both control over a process and misrepresentation of data back to operator consoles.
7.2 Security Trends in Utility Networks Although advanced telecommunication networks can assist in transforming the energy industry, playing a critical role in maintaining high levels of reliability, performance, and manageability, they also introduce the need for an integrated security infrastructure. Many of the technologies being deployed to support smart grid projects— such as smart meters and sensors can increase the vulnerability of the grid to attack. Top security concerns for utilities migrating to an intelligent smart grid communications platform center on the following trends: •
Integration of distributed energy resources
•
Proliferation of digital devices to enable management, automation, protection, and control
•
Regulatory mandates to comply with standards for critical infrastructure protection
•
Migration to new systems for outage management, distribution automation, condition-based maintenance, load forecasting, and smart metering
•
Demand for new levels of customer service and energy management
This development of a diverse set of networks to support the integration of microgrids, open-access energy competition, and the use of network-controlled devices is driving the need for a converged security infrastructure for all participants in the smart grid, including utilities, energy service providers, large commercial and industrial, as well as residential customers. Securing the assets of electric power delivery systems, from the control center to the substation, to the feeders and down to customer meters, requires an end-to-end security infrastructure that protects the myriad of communication assets used to operate, monitor, and control power flow and measurement. Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
189
Cyber security refers to all the security issues in automation and communications that affect any functions related to the operation of the electric power systems. Specifically, it involves the concepts of: • • • •
Integrity – data cannot be altered undetectably Authenticity – the communication parties involved must be validated as genuine Authorization – only requests and commands from the authorized users can be accepted by the system Confidentiality – data must not be accessible to any unauthenticated users
When designing and deploying new smart grid devices and communication systems, it´s imperative to understand the various impacts of these new components under a variety of attack situations on the power grid. Consequences of a cyber attack on the grid telecommunication network can be catastrophic. This is why security for smart grid is not just an ad hoc feature or product, it’s a complete framework integrating both physical and Cyber security requirements and covering the entire smart grid networks from generation to distribution. Security has therefore become one of the main foundations of the utility telecom network architecture and must be considered at every layer with a defense-in-depth approach. Migrating to IP based protocols is key to address these challenges for two reasons: 1. IP enables a rich set of features and capabilities to enhance the security posture 2. IP is based on open standards, which allows interoperability between different vendors and products, driving down the costs associated with implementing security solutions in OT networks. Securing OT communication over packet-switched IP networks follows the same principles that are foundational for securing the IT infrastructure, i.e., consideration must be given to enforcing electronic access control for both person-to-machine and machine-to-machine communications, and providing the appropriate levels of data privacy, device and platform integrity, and threat detection and mitigation.
7.3 Regulatory Compliance (NERC CIP) Hydro Québec is subject to regulatory security requirements that are mandated by the North American Electric Reliability Corporation (NERC). The most recently approved version of NERC CIP in 2014 is version 5. Compliance with NERC CIP standards requires comprehensive cyber security solutions including segmentation, authentication, authorization, monitoring, logging, and training; and comprehensive physical security solutions including access control, and video surveillance. The Utility Compliance solution should be based on the following design principles: •
Proper segmentation of trusted zones such as the Electronic Security Perimeter (ESP) with the use of routing, firewalling and intrusion detection/prevention technologies at each substation (CIP-005)
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
190
•
Secure remote access for Interactive Remote Access control of the ESP network (CIP-005)
•
Packet Routing at the control center with Multiprotocol Label Switching (MPLS ) and Flex VPN/DMVPN support for segmentation and encryption in the core
•
Event Correlation at the control center for monitoring and logging of events for multi-vendor network devices (CIP-007)
•
Access Control Server and Security Manager for user identity management, physical and electronic access control (CIP-004)
•
Physical access control solutions such as badge card readers; electronic door locks, controllers and sensors; physical access manager server; and video surveillance solutions (CIP-006)
The following table summarizes the various sections of the NERC CIP standard and their impact on the communication network.
NERC/CIP Section CIP-002 • Bulk Electric System (BES) Critical Cyber Asset Identification • BES Cyber System Identification • Annual Review and Approval
Solution Monitoring, Analysis and Reporting System
• • •
•
•
Date August 26, 2014
Relevant Features & Benefits Network topology views, services, links Network topology and device discovery Network Compliance Manager controlling visibility into network changes and tracking compliance with a broad variety of regulatory, IT, corporate governance, and technology best practices. Network auto-discovery with automated asset tracking including the device line cards, serial number, firmware version ensures correct complete global inventory tracking Network diagrams on demand, which will help to gain immediate and accurate insight into network relationships
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
191
CIP-003 • Security Management Controls • Cyber Security Policy (documentation and implementation) • Leadership • Exceptions • Information Protection CIP-004 • Security Awareness Program • Security Training Program • Personnel Risk Assessment Program • Access Management Program • Access Revocation Program CIP-005 • Electronic Security Perimeter • Electronic Access Controls • Monitoring Electronic Access • Interactive Remote Access Management
CIP 006 • Physical Security • Physical Security Plan • Physical Access Controls • Monitoring Physical Access • Logging Physical Work • Access Log Retention • Maintenance and Testing
• Network Admission Control • User authentication, authorization and accounting
• Enforcing security policies and compliance • Real-time audit trails including who, what, when, and why
• Security Appliances (Firewall, IPS, VPN modules) • Private VLANs • Use banners (MOTD, Login, Exec, AAA, SLIPPPP) • VPN • Netflow, IP ACLs • Wireless Point of Entry
• End-to-end security portfolio can help in building a strong Electronic Security Perimeter at all Point of Entries, including wired and wireless • Control inbound and outbound access to high security zones • Enforce strict controls over remote access of process control networks
• IP Cameras • IP Gateway Encoders • Video Surveillance manager • Hardened enclosure • Physical Cable locks
• Physical security solutions with end-toend solution for physical access monitoring from cameras, encoders to stream managers • Rugged enclosures • Cable locks for physical security • Systems to report on disconnected devices, console access etc. • Port security can detect unauthorized access of ports
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
192
• • • •
CIP-007 • System Security Management • Test Procedures • Maintenance and Testing • Ports and Services • Security Patch Management • Malicious Code Prevention • Security Event Monitoring • Documentation
• • • •
Host IDS ACS IPS Configuration assurance solution Security Monitoring Syslog Port security Event Correlation
• Security Monitoring System
CIP-008 • Incident Reporting and Response Planning • Testing & Documentation CIP-009 • Recovery Plans Specifications, Implementation and Testing
• Disaster recovery manager • Resource Manager • SAN products for disaster recovery and business continuity • Asset Management System • Network Compliance Management • Penetration Testing • Network encryption technologies for data in transit
CIP-010 • Configuration Change Management • Vulnerability Assessment CIP-011 • Information Protection • Disposal or Re-deployment
Figure 82.
• Protecting against spyware, rootkits and zero-day attacks • IPS allows for known signature detection • Security Management, Syslog can help documenting the results of a vulnerability assessment • Monitoring System, Firewall, IPS logs can help in maintaining records • Configuration assurance solution supports network configuration audits and validation • Port security disallows unwanted ports, devices • Help in characterizing and classifying cyber incidents • Monitoring System, document and report incidents • Resource manager can backup configuration files • Disaster recovery manager provides full data backup and recovery for call manager clusters
• Enforce the approval of a change in the network • Validate configurations are compliant • Identify areas for security posture improvement • Protect and secure handling of BES Cyber System Information
Network Impacts of NERC CIP v5
7.4 General Security Requirements Categorization of Cyber Assets All cyber assets need to be categorized based on their impact on the reliability of the bulk electric system. This requirement is derived from NERC CIP v5. Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
193
Furthermore, cyber assets should be mapped to named reliability service(s) that they contribute to. These services are: • • • • • • • • •
Dynamic response to bulk electric system (BES) conditions Balancing load and generation Controlling frequency (real power) Controlling voltage (reactive power) Managing constraints Monitoring & control Restoration of BES Situational awareness Inter-entity real-time coordination and communication
Security Management Control All cyber assets need to be categorized based on their impact on the reliability of the bulk electric • • • • • •
Detailed cyber security policy Management and governance Accountability and responsibility Change management Information protection Exceptions
Electronic Security Perimeter The electronic security perimeter requirements include: • • • • • •
Electronic access to cyber systems Inbound and outbound access permissions Authenticated dialup connections Detection of malicious software Remote access through intermediate device or encryption Multi-factor authentication
Patch management / Firmware Upgrade The patch management program for field devices requires: • • •
Monitoring Patch requirements (every 35 days) Locking down logical Ports & services Disabling ports and services that are not used
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
194
• • • • • •
Prevention of malicious software Account management / log logins Maintenance & testing Systems Security Management Policy-based access control Configuration change management and vulnerability assessment
Integrated Security Operations Center (ISOC) The integrated security operations center requirements include: • • • • •
Monitoring and management of both cyber and physical security threats from a centralized location Security event monitoring Cyber vulnerability assessment Incident reporting and response planning Recovery plans for bulk electric system cyber assets
Device Hardening – Intrusion Prevention (IPS) and Intrusion Detection (IDS) Capabilities Device hardening should be done as part of the base configuration of any field device. Device hardening best practice should be included in the high level design, e.g. the services that should be disabled and other general good practice configuration requirements. Intrusion detection and protection enables the identification of malformed SCADA and other data traffic. This traffic can also be removed. However, in general, utilities implement intrusion detection to enable them to identify the threat but not delete the packets. This would be deployed for example in the interconnection between the Corporate and SCADA networks. Data Manipulation Attacks The concept of data manipulation is critical because of important role that “data” play in the overall grid monitoring, automation, and control process and because the alteration of that data can either directly or indirectly manipulate that process. That network protocols can be vulnerable to attacks is well known. By sending unexpected or malformed messages that exploit bugs or inadequate defenses (e.g., buffer overflows) in protocol implementations, adversaries can crash or hijack victims. The intention of Manipulation attack is not to crash or hijack the systems but to induce other behaviors that benefit the adversaries or harm the system. “Data” here include the values of the SCADA telemetered and calculated points from generation, transmission, and distribution facilities that are reported to HMIs and SCADA consoles, where it is consumed by a human operator. Manipulation of these values can influence all aspects of the automation process. Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
195
Showing a human operator with misleading values from the field could cause the operator to override the (legitimate) automation logic, effectively sabotaging their own process. While the operator’s intentions are good, they are tricked into action through the dissemination of false data. Manipulating values used by other controllers could prevent supplementary systems—potentially including protection systems—from behaving properly.
Data manipulation can also impact higher-level operations and business functions including the manipulation of production data to influence energy trading, demand-response systems, and other back-end systems that utilize real-time energy production data. Because many of these information systems are integral to “Smart Grid” services, the manipulation of data within the process control systems of a generation facility can cascade throughout all areas of the grid. This security recommendations aim to harden the network infrastructure and its services in a highly secure environment in order to be compliant with NERC CIP Standards.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
196
8. Bibliography •
Customer Requirements Document, Faramarz Maghsoodlou, Cisco Systems Inc., and Jean Raymond, Hydro-Québec, June 16, 2014.
•
Hydro Québec and Sony Announce the Establishment of a Joint Venture to Start Development of a Large-Scale Energy Storage System for Power Grids, April 16, 2014.
•
Ultra Large-Scale Power System Control Architecture, A Strategic Framework for Integrating Advanced Grid Functionality.
•
GridWise® Transactive Energy Framework, Draft Version
•
Special Publication 800-53, Security and Privacy Controls for federal Information Systems and Organization.
•
NIST Guidelines for Smart Grid Cyber Security.
•
NPCC Regional Reliability Reference Directory # 4 – Bulk Power System Protection Criteria.
•
Designing a Reliable Power System: Hydro Québec’s Integrated Approach.
•
Hydro Québec’s Defense Plan Against Extreme Contingencies, 1999.
•
NPCC Regional reliability reference Directory # 7 - Special Protection Systems.
•
IEEE 1588, Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
197
9. Glossary Term 1PPS AAA - Authentication, Authorization, and Accounting Server AMI Bay
BES CAIDI Communications Processor
Control Control Center CT Data Center
DAU DER DFR DLMS DMS
DoS DR DSM EMS
EV
Description One pulse per second Database used for authentication of end clients. Also allows imposing access policies based on access permissions for a user or group of users. Advanced Metering Infrastructure Collection of interrelated electric power apparatus and functions that share geographic proximity and logical association. Likely to include dedicated IEDs for monitoring, control, and protection and may include a dedicated bay Ethernet switch. Bulk Electric System (term used in NERC CIP standard) Customer Average Interruption Duration Index A form of next generation SCADA RTU that may also integrate synchrophasor data, time synchronization, remote engineering access, connection to multiple SCADA masters, local HMI interfaces, and so on. An operational function used for changing and modifying, intervening, switching, controlling, parameterization and optimization of a grid asset. Hosts the applications that monitor, control, and administer the power grid. Current Transformer – Transducer Hosts the enterprise applications and services required by control center/NOC. May or may not be physically collocated with the control center. Data Acquisition Unit – A microprocessor-based device providing CT, PT, and status indication functions. Distributed Energy Resource Digital Fault Recorder – Used to store and analyze a variety of substation events. Device Language Message Specification – international standard for utility meter data exchange Distribution Management System – A suite of applications that run in the distribution control center for monitoring and control of the distribution network. The monitoring and control functions are performed through the SCADA network. Optimization is performed through various DMS advanced applications. Denial of Service Demand Response Demand Side Management Energy Management System – A suite of applications that run in the transmission control center, used by system operators to monitor, control, and optimizes the performance of the transmission system. The monitoring and control functions are performed through the SCADA network. Optimization is performed through various EMS advanced applications. Electric Vehicle
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
198
FAN FCAP
FEP GPS - Global Positioning System with Inter Range Instrumentation Group mod B (IRIG-B)
GR HMI
IED
IPSec IRIG-B LAN LOD MAIS Monitoring
MU NAT NMS
Field Area Network FCAPS, which stands for “fault-management, configuration, accounting, performance, and security” is a network management framework created by the International Organization for Standardization (ISO). Front End Processor – SCADA master front-end processor is responsible for polling information from and controlling remote SCADA RTUs. Serial time code format for providing precise time-of-day clock. Typical accuracy specification for IRIG-B: demodulated output shall be within ±100 nanoseconds (average) and ±500 nanoseconds (maximum) of UTC time. Modulated output and serial port IRIG-B shall be ±1 microsecond of UTC time. A special protection scheme for Generation Rejection Human Machine Interface – The apparatus that presents process data to a human operator, and through which the human operator controls the process. Typically provided through a graphical user interface. Intelligent Electronic Device – Various devices deployed in a substation that leverage SCADA protocols for communication. They are microprocessor-based controllers. Common types of IEDs include protective relaying devices, load tap changer controllers, circuit breaker controllers, capacitor bank switches, recloser controllers, voltage regulators, and so on. Depending on functions, they may reside in the substation control building or in a substation yard outdoor enclosure. Internet Protocol Security Inter-range instrumentation group time codes – are standard formats for transferring timing information Local Area Network Line Opening Detection A special protection scheme for automatic 735kV shunt reactor closing or tripping An operational function used for local or remote observation of a system or a process for any changes, which may occur over time. The term can also be used for observation of the behavior of a data value or a group of data values. Merging Unit – Merges data from several CTs or PTs and communicates those values to the Process Bus. Network Address Translation Network Management System – Manages the network devices (routers, switches, appliances, and so on) in the utility’s corporate network. Typically managed by the IT department. Maintains network element inventory and capabilities, performs periodic data collection from network elements, perform threshold and trend analysis. Supports central maintenance of element configurations, firmware management, security policies, and QoS policies. Fields incoming asynchronous notifications triggered by out of bounds conditions. Diagnoses root cause of a related stream of faults.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
199
NOC - Network Operations Center
NPCC OMS PAT PDC PMU
Process Bus
Protocol Stack PT PU PWE3 RLS RPTC RPTC
RTP RTS RTU SAIDI SCADA SCADA RTU SCB SLA SNTP
SOE
Station Bus
The NOC hosts applications that monitor, control, and administer the data network. May or may not be collocated with the Data Center or Control Center. Northeast Power Coordinating Council Outage Management System Port Address Translation Phasor Data Concentrator Phasor Measurement Unit – Measures voltage and current values (real and reactive) as well as frequency and angle from select points on the power system. This data is time-stamped and sent to the control center to detect anomalies in the grid. Conveys, through Ethernet LAN or serial or hardwired connection, unprocessed power system information (voltage/current samples, device statuses, etc.) from switchyard source devices such as CTs, PTs, DAUs, or MUs to the IEDs/relays that process the data into measurements and control/protection decisions. A particular software implementation of a computer networking protocol suite. Potential Transformer – Transducer Per Unit Pseudo Wire Emulation Edge to Edge A special protection scheme for Remote Load Shedding Generator Tripping & Load Shedding A special protection scheme that includes Generation Rejection (GR), Remote Load Shedding (RLS), and Remote Tripping of Shunt Reactor (RTS) Real-Time Transport Protocol is used for transferring audio across the network. A special protection scheme for Remote Tripping of Shunt Reactor Remote Terminal Unit System Average Interruption Duration Index Supervisory Control And Data Acquisition The SCADA RTU is typically a legacy system that does not support advanced processing capabilities or Ethernet/IP interfaces. Series Compensation Bypass Service Level Agreement Simple Network Time Protocol – SNTP is a networking protocol for clock synchronization between computer systems over packet-switched, variable latency networks. Sequence of Events Recorder – Produces chronological list of when monitored devices change state for post-disturbance analysis. This requires 1ms of accuracy. Interconnects IEDs, distributed controllers, and HMI. Provides the connection from these devices to the WAN router. May also provide direct connectivity for DFR, SCADA RTUs, communication processors if a distributed controller is not present.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
200
Substation Control Building
Substation Yard TOU UFLS UVLS WAN
Resides in the substation location. Used to host the Station Bus, multiservice bus, and IEDs and other substation devices that interface to the Station Bus. An extension of the substation control building hosting devices that are connected back to the Control Building IEDs via the Process Bus. Time of Use A special protection scheme for Under Frequency Load Shedding A special protection scheme for Under Voltage Load Shedding Wide Area Network – The WAN is the primary link from the substation to the energy control center with optional backup link for redundancy. The WAN may be leased network capacity or a private network owned and operated by the utility.
Date August 26, 2014
Use Cases Proposal for IEC © 2014 Cisco Systems, Inc.
201
View more...
Comments
