Yes, large-scale data can be stored on the Ethereum blockchain
August 13, 2019•874 words
WePower/Elering Nationwide Energy Experiment
Hinsights from WePower website:
WePower’s mission is to enable everyone to make a change towards a sustainable energy future through their energy purchasing decisions. WePower essentially tackles the limitations of the current instruments that exist in the market, especially Power Purchase Agreements (PPAs), which are too complex and expensive for corporate energy buyers. That's why WePower has built a next-generation renewable energy procurement and trading platform based on smart contracts that uses virtual PPAs.
...so we have #blockchain for #P2P transactions and #transparency in the energy market, #SmartContracts for cheaper, faster, transactions... and on top a dedicated #token.
Sounded like nothing new or worth further reading. Not the first time people claims revolutionizing some market thanks to Blockchain.
Until I read about the test carried WePower carried out. They don't just throw out buzz words. They did perform a real test: trying to write on the Ethereum blockchain energy production and consumtion data on a national scale.
And they did it succesfully.
Scope
1 year of actual production and consumption data was provided by the estonian energy operator Elering
Volume of data points
households | 7x10⁵ |
timeframe | 1 year |
readings | 1/hour/household |
7x10⁵ x 365 x 24 = 6.132x10⁹
Time
WePower assumed they could store 200 data points per block.
Total volume | 6.132x10⁹ |
Assumed volume per block | 200 |
Seconds per block | 15 |
6.132x10⁹ * 15 / 2x10² = 4.599x10⁸ s ~ 5323 days ~ 15 years
Costs
Based on the Ethereum yellowpaper (page 20)
Store operation | 2x10⁴ gas cost |
Every Transaction | 2.1x10⁴ gas cost |
Average Gas Price | 8.5 Gwei |
1 ETH | 1x10¹⁸ wei |
1 ETH | 185.68 € (coinmarketcap, 13th August 2019 |
6.132x10⁹ x (2x10⁴ + 2.1x10⁴ / 200) * 8.5x10⁹ = 1.04791281x10²⁴ = 104,791.281 ETH = 195 M€
--> WePower needs 14 years and 195 M€ to run their pilot/test... GAME OVER? Not quite yet.
Optimization
To cut down testing costs and time, they:
- compressed data
- EVM uses 32 bytes per words. Or numbers represented by 256 bit integers, which was more than required for one point. They figured they could fit 15 data points per 256 bits. Meaning 15 times less gas costs and 15 times less validation time.
- reduced the volume of data by:
- aggregating at zip code level --> 3837 points (also added benefit of anonymizing data and complying with GDPR regulations)
- summarizing hourly consumption into monthly consumption -->
3837 x 12 = 46,044 data points
Test results
Stats
Total Transactions | 434 |
Gas Used | 1,129,171,462 |
Average gas price (Gwei) | 9.65921659 |
Median gas price (Gwei) | 2.5 |
Cheapest gas price (Gwei) | 1.5 |
Highest gas price (Gwei) | 60 |
Average wait time (s) | 1,456.5 |
Median wait time (s) | 203.6 |
Shortest wait time (s) | 0.8 |
Longest wait time (s) | 44,882.1 |
Takeaways
Higher gas price doesn't necessarily means shorter confirmation times
Chart - Median time to transaction confirmation per gas price
With the intention of reducing testing time and costs, they were batching transactions by 15 per block. This actually led to longer waiting times when "gas price increased while transaction were still waiting to be confirmed. Other transactions from the batch had to wait in line until the gas price would meet again the set price level". (see transactions at gas price 4 Gwei).
"Performance depends on other activities being performed at the same time on the blockchain, meaning that increasing gas price does not always guarantee timely confirmation of transactions."
Speed was achieved in being reactive, meaning readjusting the set gas price frequently enough to avoid being below the market price. Practically, they ended readjusting the gas price every 3 transaction using ETH Gas Station API
Gas price equal or lower than the average secures confirmation time lower than 5 minutes
Chart - Distribution of transactions per gas price strategy
- Safe low: both cheap and successful. Lowest price where at least 5% of the network hash power will accept it.
- Average: accepted by top miners who account for at least 50% of the blocks mined- safe and prompt. Usually reflects wallets' defaults.
- Fastest: lowest gas price accepted by all top miners. Should secure transactions will be accepted by all the top pools. Paying more than this price is unlikely to increase transaction confirmation time under normal circumstances.
Chart - Transaction confirmation time distribution
Reading both charts, we see that while ~85% transactions were submitted using an "average" or worse (safelow) price, ~60% transactions were still confirmed in less than 5 minutes.
For the energy market, my personal guess is that this confirmation time is satisfactory.
Conclusion
This test was successful both for WePower and the Ethereum network.
One one hand the pilot "helped WePower to validate and verify the logic and processes that will be at the core of the WePower platform.": they can go forward developing their platform with confidence.
On the other hand their test confirmed that "Ethereum is mature enough to accommodate contracts with multi-year terms." WePower ends their report acknowledging "that the scalability of Ethereum blockchain currently has limitations" but also reminds that "the problem is being tackled by Ethereum developers with plans to implement sharding, i.e. partitioning data into subsets, and moving from energy-intensive Proof of Work to a more environmentally friendly Proof of Stake consensus model".