the uk-dale dataset, domestic appliance-level electricity demand and whole-house demand from five uk homes - residential energy storage

Smart meters are being introduced in many countries.
These indicators measure the total electricity demand of a household.
However, research into consumer behavior shows that consumers are best able to improve energy efficiency when providing item-by-item appliancesby-
Household appliances consumption information.
Energy decomposition is a computing technology for estimating equipment. by-
Overall energy consumption of household appliances
House meter signal
In order to study the decomposition algorithm, the researchers need not only to describe the data of the total demand of each building, but also to describe the "basic truth" requirements of a single device.
In this regard, we introduced the United KingdomDALE: an open-
Access data set from UK audio recording home appliances-
Taking the sampling rate of 16 khz as the whole-
House for personal appliances and 1/6 hz.
This is the first open access UK data set at this time resolution.
We recorded from five houses, one of which was recorded for 655 days, the longest duration of any energy data set we know at this sample rate.
We also described the low. cost, open-
The wireless system we built to collect data sets.
Prudent management of electricity consumption is becoming more and more important.
However, research on residential energy users shows that the vast majority of people are estimating their overall
Residential Energy Consumption or energy consumption of individual equipment.
Residents often underestimate the energy used for heating, and overestimate the consumption of obvious equipment such as lighting and television.
The failure of residents to properly estimate energy consumption is likely to lead to an increase in total consumption.
How important is occupant behavior in determining total energy usage?
In the same house, energy use can be two or three times different, and similar electrical appliances are occupied by people with similar populations.
These huge differences in energy consumption are due to differences in consumer behavior.
If the House provides better feedback on which devices use the most energy, then users can adjust their behavior to use the appliance more efficiently.
Smart meters are such a feedback mechanism.
The British government has asked energy retailers to install smart meters by 2020. The roll-
Already outside.
Similar smart meter roll-
Many countries plan to go out.
The business case for smart meters in the UK assumes that smart meters will save £ 4.
6 billion reduced energy consumption (
Electricity and gas included).
Smart meters only provide energy consumption measurements for the entire house, but behavior studies show that consumers are best able to manage their electricity consumption when given information.
Energy decomposition is designed to estimate equipment-by-
The equipment consumption of smart meter signals may therefore play an important role in realizing the energy saving of smart meter business case prediction.
Energy decomposition is an active research field (
See Armel for recent comments).
Researchers need to access large data sets recorded in the field to develop decomposition algorithms, but it is not practical for each researcher to record their own data sets.
Therefore, the creation of open access data sets is key to promoting a dynamic research community.
Researchers at the Massachusetts Institute of Technology released a reference Energy decomposition data set (REDD)in 2011 ()
Subsequently, researchers from the United States, Canada, India, France, the United Kingdom, Switzerland, Portugal, Italy and Austria released more data sets.
In order to test the performance of a country-specific decomposition algorithm, it is necessary to be able to obtain data for that country, because there is a large difference in electricity consumption between countries;
Because different countries use different devices, and because different cultures display different usage patterns.
When writing, the only open
The access data set recorded in the UK is the DECC/DEFRA home power study with a sample period of two minutes.
However, this sampling rate is 12 times slower than the UK smart meter, which is sampled every 10 seconds.
It is this kind of smart meter data that will provide input for the decomposition algorithm, so researchers need to access the 10 m/s data to design the decomposition algorithm for the UK (
Other countries will also use smart meters with similar sampling cycles).
We show the first open access UK data set with high time resolution.
We recorded it from five houses.
We record the active power of a single device and the entire device every six seconds.
Apparent electricity demand for houses.
In addition, in three houses, we have a whole
House voltage and current at 44. 1u2009kHz (down-
Sample to 16 khz for storage)
Active Power, apparent power and RMS voltage at 1 hz are also calculated.
In House 1, we recorded 655 days and recorded them separately from almost every electrical appliance in the house, resulting in records of 54 separate channels (
Although there are fewer channels recorded at the beginning of the data set).
We will continue to record from this House for the foreseeable future.
We recorded from four other houses for several months;
Each of these houses recorded 5 to 26 single device data channels.
Provides an overview of the system design and a summary of the data set.
This data set may also be useful for researchers working on the following: the ideal properties of the data set used for decomposition include: we first describe how we monitor a single device every 6 kbps s and then how we record the entire device
The main power supply of the house on the 44 th. 1u2009kHz.
Houses in the UK, like houses in our data set, the main "ring" extends out of the fuse box.
Many sockets may share the same ring.
So in order to measure a single device in the UK, we have to install the plugin
In a single device monitor (IAMs)
Between each device and its wall socket.
We used the EcoManager transmitter plug developed by the current cost and distributed by EDF Energy.
The standard base station for these IAMs is EcoManager.
EcoManager can only handle up to 14 transmitter plugs and can only provide data once per minute through its serial port.
We need up to 54 appliances per house and data is needed every 10 m/s or faster.
In order to achieve this goal, we began to build our own base station.
With the help of others in the community (
See thanks)
We reconstruct the specification of the EcoManager protocol.
With the reconstruction protocol, we program an open base stationsource, rapid-
A development platform called Nanode.
16 MHz Nanode includes those running on the ATmega328P Atmel micro-controller (
Several ardunos are used on the same micro controller)
RF with HopeRF RFM12b (RF)module.
The EcoManager product seems to be using the same (or similar)
RF modules tuned to the 433 mhz ISM band (
Please note that it is illegal to use the 433 mhz band without permission in some countries).
We configured packets from the Serial Peripheral Interface sniffing using bus piracy, finding the right starting point for our RF configuration settings (SPI)
Connect the micro-controller of EcoManager to its RF module.
Each IAM chooses its own 32-
When the "right" button of IAM is pressed, the bit ID is random.
Each IAM stores its ID in a non-Unstable memory
Our base stations maintain a list of these IDs and poll each IAM in order.
Although the power data in this packet may have a few seconds history, each IAM replies to its polling packet in 20 milliseconds.
The EcoManager RF protocol uses modular and checksum bytes to provide some flexibility against RF corruption.
Power data is sent from our Nanode base station via FTDI to the data recording PC-to-USB cable.
It is also possible to turn IAMs on or off remotely.
Measure power from hard
We use current cost transmitters such as boiler and kitchen ceiling lamps (TX)
Transformer with current (CT)clamps.
These transmitters use the same radio frequency as the EDF IAMs, but use different protocols.
In particular, the current cost transmitter is unable to receive RF data.
Instead, they transmit packets every 6 + 0.
3 seconds without first checking if the RF channel is clear.
Therefore, the RF conflict is inevitable and there is no mechanism for re-entry
Transfer the lost data.
In this way, our base station learns the transmission cycle of each current cost TX and ensures that the base station does not transmit in a short period of time until it is expected to arrive, thus minimizing the chance of packet conflicts the current cost of TX packets.
Checksums are not used for current cost transmitters.
Instead, they use Manchester coding.
RF corruption may result in invalid Manchester code.
If this happens, the receiver can detect the corrupted Manchester code and discard the packet.
Unfortunately, in the event that the Manchester encoding is not invalid, corruption may damage the data payload, so the corrupted packet cannot guarantee detection from the current cost transmitter.
To maximize the distance we transmit, we have tried several antenna and RF module configurations.
We set a quarter.
Wavelength antenna combined with the ground plane composed of four quarters-
The cross-shaped wavelength line runs on the ground plane, coming from directly below the point at which the antenna is connected to the Nanode printed circuit board.
Given that smart meter data will flood in the near future, decomposition researchers need to access data as similar as the data recorded by smart meters in the near future.
Unfortunately, the installation or acquisition of "real" smart meters is not simple: the installation requires an electrician from a utility company and, crucially, the current UK smart meter engineering specification has not yet been finalized (
SMETS2 documentation is not an engineering specification).
UK gas has installed more than a million "real" smart meters, but these do not fit SMETS2 and none of our participants have 1 m installed.
So we have to build our own measurement system.
The current iteration of the UK smart meter specification is detailed enough to allow us to build our own metering system that is very similar to what the UK smart meter may provide (
These specifications are subject to the formal change control process, so any changes are subject to analysis and approval from stakeholders).
Latest specifications, smart meters are required to connect to the home Lan (HAN)
Use ZigBee Smart Energy protocol v1.
It is speculated that the decomposition system will access the smart meter data through the "consumer access device (CAD)
South Korea is connected.
CADs can request instantaneous active power and pricing data to the smart meter every 10 seconds.
In addition, CADs will be able to extract other data, such as half an hour of 13 months to actively import data, and half an hour without reaction import data for 3 months and a half hours without reaction export data for 3 months [SMETS2 ()
Personal communication with DECC, October 2013].
We started to build a metering system that collects active power once a second, and sampled voltage and current waveforms at 44.
1 khz allows researchers interested in this high frequency data to use it.
One solution is to use off-the-
Current cost of the entire shelf
Household transmitter with current transformer (CT)clamp.
These work with our wireless base stations.
We used this solution in several houses where our custom solutions were impractical.
There are several drawbacks to using a CT fixture connected to a wireless transmitter: We know that there is no existing home energy monitor that provides an accurate proxy for smart meters in the UK.
Expensive power quality monitors, which cost hundreds or thousands of pounds, can be measured with the precision we need, but these are very expensive and some require CT sensors, a split core, therefore, the installer is required to disconnect the tail of the meter from the utility's meter, which can only be done with the permission of the utility.
We raised a low.
High cost, high resolution, easy to install recording technology
Home Power requirements using computer sound cards, CT clips and AC power suppliesAC adapter.
A typical sound card has a very good analog-to-digital converter (ADCs).
Typical specifications for modern sound cards include: in order to record the power supply voltage and current waveform, we need a simple circuit to connect the sound card to the AC power supply
AC adapter and CT fixture ().
The circuit does not require the user to handle any dangerous voltage.
We used this line.
The input of the sound card, not the input of the microphone, because the line-
Input should provide a lower noise signal path than the front of the sound card microphoneamplifier.
Maximum standard peak-to-
Peak voltage of consumer audio device line-input is 0. 89 volts.
Therefore, the purpose of our circuit must be to reduce the output voltage of each sensor so that we do not provide a voltage above 0.
The voltage of the sound card is 89 volts.
In order to measure the mains voltage as safely as possible, we used standard AC-AC adapter (
Ideal Power 77DB-06-09’).
This provides an open peak
Circuit output voltage of about 11 volts.
The research done by the open energy monitoring project shows that communication-
The AC adapter should track the power supply input voltage linearly in the range of 185. 5 to 253u2009V.
We reduced the air conditioning.
AC adapter output voltage with voltage divider circuit (
We used two resistors: 10 kk Ω and 20 Ω)
Production is about 0. 7u2009V peak-to-
Enter the peak in one channel entered into the sound card line.
In order to measure the mains current, we used a current transformer (CT)clamp (the ‘YHDC SCT-013-000’).
CT clip and AC-
The AC adapter comes from the open energy monitoring store.
The CT is connected in parallel with the 22 Ω load resistor.
This configuration generates about 0. 89u2009V peak-to-
When the primary current of the CT is 30 amps RMS, the peak on the load resistance, we think this is the maximum current that any house that is being studied will pull.
To prevent the sound card from being overloaded, both channels include a fast 80 ma
Fuse and a pair of 1N5282 diodes (with a 1.
Forward voltage bias)
To ensure that more than 1 circuit cannot be provided.
3 v of sound card.
Let's calculate a rough estimate for our measurement resolution.
If we want to measure the primary current with a range of 0 to 30, then we should be able to solve the change of the primary current of about 3 mA per sample (
2ADC steps and 3mA).
For voltage measurement, if we want a range of 0 to 253 V (230V + 10%)
Then, we should be able to solve about 22 mV of the changes per sample (and ÷).
Considering that the sensor can be noisy, and we only provide 0.
For the ADC of the voltage measurement, we should reduce the resolution of each sample to about 30 mV and 5 mA of the voltage and current respectively.
This gives us a power resolution of about 30 m × 5ma = 150 mW.
We now describe the software for our sound card watt-hour meter.
We use the following relationship to calculate | (apparent power)and (
Power of "real" or "active)
Record vectors at the same time as reading from voltage and current (
We record in the form of blocks, the duration of each block is 1 u2009 s;
This time period is chosen because REDD uses this sample time period for master data)
: The sum is the root mean square value of the current and voltage vectors respectively;
Is the number of samples;
Are the first few samples of current and voltage vectors, respectively.
The system cannot guarantee that we always process blocks whose length is equal to the exact integer times of the main cycle period, but, as shown in the technical verification section, our relative error is always less than 2%.
The conclusion is that the resolution we get is greater than the resolution needed to provide a good proxy for a "real" smart meter (
Although we acknowledge, we do not know the exact resolution of the "real" smart meter.
This decision is likely to be left to the manufacturer [
Personal communication with DECC on March 2013]).
We save it once per second with the precision of 2 decimal places in the CSV file, | and save it to disk.
We also save the original ADC data to disk.
ADC data is turned off in order to reduce the required space
Use open-
SOURCE Audio tools sox to 16 khz (
REDD uses 15 khz, we originally wanted to use the standard defined by REDD, but we found that 16 khz support is more common in processing tools than 15 khz). The ADC is 20-
But there are very few audio processing tools that can handle 20-
Bit file, so we populate each sample to generate 24-bit file.
Uncompressed 16 khz 24-
The bit File needs 28.
8 bytes per day, so we compress files using a Free Lossless Audio Codec (FLAC)
Reduce storage requirements to about 4.
8 gb per day.
In order to convert the original ADC value to a voltage and current reading, we must first find the appropriate conversion constant.
We calibrate each data acquisition system separately to compensate for the manufacturing variability of the components.
When each system is first installed, we calibrate it once.
Let's connect watts?
During setup, professional meters are provided to the data recording PC via USB to automatically calibrate the voltage and current conversion coefficients.
We usually use a resistance load like a kettle to calibrate the system.
If it's wattage?
The power factor reported by the meter is greater than 0.
97 then, the calibration script also calibrated the phase offset introduced by the sensor.
We implemented the power monitoring system described in five software projects.
All packages are available from name>.
The package is: r: Nanode C code.
This code allows Nanode to talk directly to multiple current costshouse sensors (CC TXs)
And multiple EDF transmitter plugs (CC TRXs).
The user speaks to Nanode via a serial port.
The user sends simple commands.
It sends the data back to the PC in a simple JSON format.
Python script to communicate with rfm_edf_ecomanager Nanode system.
This provides a command.
Line tool for "pairing" sensors with logging systems;
Allocate manpower
The readable names of these sensors are then recorded to the disk in the CSV file using the same format as the REDD file for MIT.
The focus is on reliable logging.
Rfm_ecomanager_logger Nanode if you try to restart the crash.
Rfm_ecomanager _ logger ensures that the timestamp is correct as much as possible (
Considering that Nanode does not have a real-time clock and that serial data can be saved in the buffer of the operating system if the system loads heavily, it doesn't matter).
Each channel records data about once every six seconds.
Generate statistics and charts from REDD
Format the power data.
It is mainly used to detect the health of sensors.
Each log system has a Python module for "recording ".
Send an email if the sensor stops working or rfm_ecomanager _ logger fails.
Also send a daily "heartbeat" email to the homeowner with statistics (
By powerstats)
Describe the power data for the last day.
Useful "health" information about the system, such as the remaining disk space, is also provided.
A system used to record the voltage and current waveform at 44. 1u2009kHz, 20-
Use each channel bit of the PC sound card.
Calculate the CSV file once per second and save the active power, as in power, and RMS voltage to the CSV file. Records down-
Sample the ADC data into the FLAC file.
In order to collect our own data sets, we installed the following equipment in each house: the system diagram is shown in the figure.
The CSV data file recorded by the data recording PC is transferred to the remote server every morning using rsync.
Use external hard drives to manually transfer backhand files every two months.
The subjects of study are either master's or doctoral students from Imperial College.
Subjects chose to do a research project with the author.
Help with their own projects and collections in the UK
With the Dell data set, students kindly agreed to install metering hardware in their house.
The upper limit on the number of houses we can record is made up of a limited fiscal budget, limited time to assemble metering hardware, and limited the number of students who voluntarily engage in research projects related to domestic energy consumption.
In each house, the owner chooses the electrical appliances to be recorded, and the author suggests that the most energy-efficient electrical appliances should be recorded first.