Understanding the data integration requireements and options at WindESCo
Data integration refers to the process of integrating with the your data collection source. This is a process that can be quite complex as high-speed data is still a relatively new aspect of the wind industry. It is particularly helpful when WindESCo can review a data file sample to confirm signals available, the signal frequency, and if there are any unique mapping considerations for the data team.
What is included?
- Establishing a data integration connection with the customer
- Support in basic troubleshooting and configuration with the WindESCo format
- Ingestion of data into the WindESCo system
Required data?
Up to one year of a combination of:- High-frequency data (<30sec) for ~30 signals
- 10-min data (all available signals)
- Event data; Note: if event data is not provided, recommendations will be severely limited
WindESCo prefers data as it is stored, not re-sampled to be at a lower frequency. Some checks require a baseline amount of high-frequency data to be able run. The longest requirement is yaw misalignment which requires 3 months of data before the first assessment is completed. Hence, if you would like all checks to immediately run upon data integration, you would provide:
- Ideally: 1 year of high-speed data
- Alternatively, 1 year total of 10-minute and high-speed data that is available. For example, 3 months of high and 9 months of 10-min data to get up to a year
- If no high-speed data is available, we set up a system to collect real-time data and wait 3 months to get this
- Additionally, the log of event data - informational event data and alarms for each turbine for 1 year
Types of Data Integration Supported?
- API (PI, Bazefield, Greenbyte)
- Database (MS SQL, MySWL, PostGRES)
- File Export
- WindESCo's data collection via virtual or physical WeEdge server
WindESCo is continuously adding data integrations that we support, please don't hesitate to reach out with questions!
Option 1: historical high-speed data retrieved from API
Example APIs include PI and Bazefield. Other APIs are possible as well. WindESCo will run an import script from AWS cloud infrastructure to obtain historical data, then run a script on a daily basis to keep our repository up to date with the latest data. Analytics can run as soon as the initial import is complete.
Requirements:
- Customer API is publicly accessible and secure (uses encryption and authentication)
- Customer provides credentials to WindESCo to authenticate
- If a PI API is used, Basic Authentication is the method of authentication
Option 2: historical high-speed data retrieved from database
Example databases include MS SQL Server, MySQL, and PostGRESQL. WindESCo will run an import script from AWS cloud infrastructure to obtain historical data, then run a script on a daily basis to keep our repository up to date with the latest data. Analytics can run as soon as the initial import is complete.
Requirements:
- Customer database is publicly accessible
- Customer provides credentials to WindESCo
- Database is updated continuously, or at least regularly (e.g. each day)
- Database is performant (e.g. data for 1 turbine for 1 day retrieved in 10 seconds or less)
Option 3: historical high-speed data exported to files and transferred to WindESCo
If direct access to the high-speed historical data can not be given to WindESCo, then it will be necessary to export data to files and transfer the files to WindESCo. Options for file transfer include:
- SFTP server
- AWS S3
- Azure
WindESCo will run an import script from AWS cloud infrastructure to obtain historical data files, then run a script on a daily basis to keep our repository up to date with the latest data. Analytics can run as soon as the initial import is complete.
Requirements:
- Customer provides a list of available signals
- Customer provides a sample data file (e.g. one turbine for one day)
- After initial import, customer puts exported files in repository regularly (e.g. each day)
- Customer provides credentials to WindESCo to access repository
- Data file format according to “WindESCo Data File Format Specification”
Option 4: historical 10-minute, and real-time data retrieved from Greenbyte API
The Greenbyte API is similar to those described in Option 1. However, Greenbyte does not typically have historical high-speed data. It is worth asking the customer as Greenbyte is starting to store some HF data (by request / fee) for customers.
Instead, a logger application is set up by WindESCo, and the real-time endpoint is polled to get the latest data. This requires data to be collected for 3 months before WindESCo can provide analytics results. The Greenbyte API is also used to import historical 10-minute data, and possibly event data.
Requirements:
- Customer provides an API token to WindESCo
Option 5: High-speed data collected by WeEdge in real time
When historical high-speed data is not available, it is necessary to set up a system to collect high-speed data in real time. WindESCo requires a Windows PC (called “WeEdge”) that can access the SCADA interface, usually at the wind plant. Standard options for the SCADA interface include:
- OPC XML DA
- OPC UA
- Modbus TCP
Other SCADA interface options are possible but may include a surcharge.
- OPC DA
- IEC-60870-5-104
Data must be collected for 3 months before WindESCo can provide analytics results.
Requirements:
- Customer provides a list of available signals
- Customer provides a Windows PC per specification. WindESCo installs software on PC.
- Customer provides remote access to WeEdge for up to two WindESCo staff
- Customer configures network per specification
Options:
- WindESCo can supply and industrial PC that can be rack or wall mounted
In addition to three months of high-speed data, WindESCo also needs one year of 10-minute historical data. These are typically provided by files. See Option 3 for a description of the file format, transport, and data partition requirements.