Yes, large-scale data can be stored on the Ethereum blockchain

WePower/Elering Nationwide Energy Experiment

Hinsights from WePower website:
WePower’s mission is to enable everyone to make a change towards a sustainable energy future through their energy purchasing decisions. WePower essentially tackles the limitations of the current instruments that exist in the market, especially Power Purchase Agreements (PPAs), which are too complex and expensive for corporate energy buyers. That's why WePower has built a next-generation renewable energy procurement and trading platform based on smart contracts that uses virtual PPAs.

...so we have #blockchain for #P2P transactions and #transparency in the energy market, #SmartContracts for cheaper, faster, transactions... and on top a dedicated #token.
Sounded like nothing new or worth further reading. Not the first time people claims revolutionizing some market thanks to Blockchain.
Until I read about the test carried WePower carried out. They don't just throw out buzz words. They did perform a real test: trying to write on the Ethereum blockchain energy production and consumtion data on a national scale.

And they did it succesfully.

Scope

1 year of actual production and consumption data was provided by the estonian energy operator Elering

Volume of data points

households 7x10⁵
timeframe 1 year
readings 1/hour/household

7x10⁵ x 365 x 24 = 6.132x10⁹

Time

WePower assumed they could store 200 data points per block.

Total volume 6.132x10⁹
Assumed volume per block 200
Seconds per block 15

6.132x10⁹ * 15 / 2x10² = 4.599x10⁸ s ~ 5323 days ~ 15 years

Costs

Based on the Ethereum yellowpaper (page 20)

Store operation 2x10⁴ gas cost
Every Transaction 2.1x10⁴ gas cost
Average Gas Price 8.5 Gwei
1 ETH 1x10¹⁸ wei
1 ETH 185.68 € (coinmarketcap, 13th August 2019

6.132x10⁹ x (2x10⁴ + 2.1x10⁴ / 200) * 8.5x10⁹ = 1.04791281x10²⁴ = 104,791.281 ETH = 195 M€

--> WePower needs 14 years and 195 M€ to run their pilot/test... GAME OVER? Not quite yet.

Optimization

To cut down testing costs and time, they:

  • compressed data
    • EVM uses 32 bytes per words. Or numbers represented by 256 bit integers, which was more than required for one point. They figured they could fit 15 data points per 256 bits. Meaning 15 times less gas costs and 15 times less validation time.
  • reduced the volume of data by:
    • aggregating at zip code level --> 3837 points (also added benefit of anonymizing data and complying with GDPR regulations)
    • summarizing hourly consumption into monthly consumption --> 3837 x 12 = 46,044 data points

Test results

Stats

Total Transactions 434
Gas Used 1,129,171,462
Average gas price (Gwei) 9.65921659
Median gas price (Gwei) 2.5
Cheapest gas price (Gwei) 1.5
Highest gas price (Gwei) 60
Average wait time (s) 1,456.5
Median wait time (s) 203.6
Shortest wait time (s) 0.8
Longest wait time (s) 44,882.1

Takeaways

Higher gas price doesn't necessarily means shorter confirmation times

Chart - Median time to transaction confirmation per gas price

With the intention of reducing testing time and costs, they were batching transactions by 15 per block. This actually led to longer waiting times when "gas price increased while transaction were still waiting to be confirmed. Other transactions from the batch had to wait in line until the gas price would meet again the set price level". (see transactions at gas price 4 Gwei).
"Performance depends on other activities being performed at the same time on the blockchain, meaning that increasing gas price does not always guarantee timely confirmation of transactions."
Speed was achieved in being reactive, meaning readjusting the set gas price frequently enough to avoid being below the market price. Practically, they ended readjusting the gas price every 3 transaction using ETH Gas Station API

Gas price equal or lower than the average secures confirmation time lower than 5 minutes

Chart - Distribution of transactions per gas price strategy
  • Safe low: both cheap and successful. Lowest price where at least 5% of the network hash power will accept it.
  • Average: accepted by top miners who account for at least 50% of the blocks mined- safe and prompt. Usually reflects wallets' defaults.
  • Fastest: lowest gas price accepted by all top miners. Should secure transactions will be accepted by all the top pools. Paying more than this price is unlikely to increase transaction confirmation time under normal circumstances.
Chart - Transaction confirmation time distribution

Reading both charts, we see that while ~85% transactions were submitted using an "average" or worse (safelow) price, ~60% transactions were still confirmed in less than 5 minutes.

For the energy market, my personal guess is that this confirmation time is satisfactory.

Conclusion

This test was successful both for WePower and the Ethereum network.
One one hand the pilot "helped WePower to validate and verify the logic and processes that will be at the core of the WePower platform.": they can go forward developing their platform with confidence.
On the other hand their test confirmed that "Ethereum is mature enough to accommodate contracts with multi-year terms." WePower ends their report acknowledging "that the scalability of Ethereum blockchain currently has limitations" but also reminds that "the problem is being tackled by Ethereum developers with plans to implement sharding, i.e. partitioning data into subsets, and moving from energy-intensive Proof of Work to a more environmentally friendly Proof of Stake consensus model".


You'll only receive email when they publish something new.

More from sripwoud
All posts