Today one of the rare solar eclipses has taken place. In this community there were fantastic posts preparing us for the eclipse; Vitaliy Kaurov showed some nice image processing of photos of the eclipse. Solar eclipses have always fascinated people: in ancient times they were often feared. They were often supposed to be a bad omen and sometimes they actually were, for example for two astronomers who were beheaded on the order of Emperor Zhong Kang, because they had failed to predict a solar eclipse. Even before todays eclipse there were reports warning that Europe might face large scale black outs, because the power grids would be strained by a lack of solar power.

This is why I decided to use Mathematica to analyse some data about the effects on the power grid in the UK. I also use data from the Netatmo weather station to analyse variations in the temperature in Europe due to the eclipse. Let's start with the temperature data. Aberdeen is quite close to the region of the total solar eclipse:

so we can hope to see some effect. First, we need to find out when the solar eclipse took place in Aberdeen (UK)

where I live.I download the respective data from a website:

Import["http://www.timeanddate.com/eclipse/in/uk/aberdeen", "Data"][[2, 2]] // TableForm

This gives

So the eclipse started at about 8:30am in the morning and ended at 10:46am. I will use this to mark the relevant time intervals in some of the images in this post. Next, I need data on the temperature. I own a Netatmo weather station

that measures temperature, humidity and pressure every five minutes or so. I can download that data from my account. The corresponding csv file is attached to this post (Outdoor203_2015.csv). I use Mathematica 10's new SemanticImport

tempnetatmopriv = SemanticImport["~/Desktop/Outdoor_20_3_2015.csv"]

which gives

We can plot the data:

DateListPlot[Lookup[tempnetatmopriv // Normal, {"column2", "column3"}][[4 ;;]], Filling -> Bottom, 
 Epilog -> {{Red, Opacity[0.2], Rectangle[{DateList[{2015, 3, 20, 8, 33, 0}], 6}, {DateList[{2015, 3, 20, 10, 46, 0}], 9.8}]}}, 
 AspectRatio -> 0.25, ImageSize -> Full, FrameLabel -> {"Time", "Temperature degrees centigrade"}, LabelStyle -> Directive[Bold, Medium]]

The red box marks the time of the eclipse and a clear drop in temperature shows the reduced solar irradiation. Of course, this could be a very localised event, and is not sufficient evidence for an effect of the eclipse. I therefore use the public data feature of Netatmo, which I described in an earlier post. The main idea is that I can use an API to get data from many stations in all Europe. I used the same netatmo.sh file descried in the earlier post, and wrapped in into a RunScheduledTask:

m = 1; RunScheduledTask[Run["~/Desktop/netatmo.sh"];
 data = Import["https://api.netatmo.net/api/getpublicdata?access_token=" <> 
 Last[StringSplit[Import["~/Desktop/request-token.txt", "CSV"][[1, 1]], "\""]] <> "&lat_ne=59.91&lon_ne=13.75&lat_sw=40.42&lon_sw=-20.0&filter=\
True", "Text"]; tab = Quiet[Select[Select[Table[ToExpression /@ Flatten[StringSplit[#, "]"] & /@ StringSplit[#, "["] & /@ If[Length[
 StringSplit[StringSplit[data, "place"][[k]], ","]] > 12, Drop[StringSplit[StringSplit[data, "place"][[k]], ","], {5}], 
           StringSplit[StringSplit[data, "place"][[k]], ","]]][[{2, 3, 7, 8, 15}]], {k, 2, 
       Length[StringSplit[data, "place"]]}], Length[Cases[Flatten[#], $Failed]] == 0 &  ], Length[#] == 5 &]];
  Export["~/Desktop/Databin/DataTemp" <> ToString[1000 + m] <> ".csv", {DateString[], tab}]; m++, 900]

This function already cleans the data a wee bit and exports it to a data folder. I did this locally, but of course the new Wolfram Databin is a very useful feature here, too. Anyway, I can now Import all data:

Monitor[netatmoeurope = 
   Table[Import["~/Desktop/Databin/DataTemp" <> ToString[1000 + m] <> ".csv"], {m, 1, 91, 2}];, m]

and plot it into individual frames:

Monitor[For[m = 1, m <= 46, m++, 
  datanetatmo = {netatmoeurope[[m, 1]], ToExpression@netatmoeurope[[m, 2]]};
  Export["~/Desktop/SolarEclipseFrames/frame" <> ToString[1000 + m] <>".gif", 
   GeoRegionValuePlot[GeoPosition[{#[[2]], #[[1]]}] -> #[[3]] & /@ datanetatmo[[2]], 
   PlotRange -> {-7, 30}, ColorFunction -> "TemperatureMap", ImageSize -> Full, 
    Epilog -> Inset["Time: " <> datanetatmo[[1]], {-0.3, -0.3}, BaseStyle -> Directive[Large, Bold]]]]], m]

I am on a Mac and use the command

convert -delay 35 -loop 0 frame*.gif animated.gif

to generate the following animation:

It is a bit difficult to see but around frame 20 the temperature drops a bit, particularly in the nordic countries. This is actually easier to see when we look at a differentiated time series. The following plot represent the temperature change in several location in Europe:

Monitor[netatmoeuropedata = 
   Table[Import["~/Desktop/Databin/DataTemp" <> ToString[1000 + m] <> ".csv"], {m, 1, 91, 2}];, m]

and then

ArrayReshape[Table[Differences@(ToExpression@#[[2, n]] & /@ Delete[netatmoeuropedata, 14])[[All, 3]] // ListLinePlot, {n, 1, 9}], {3, 3}] // TableForm

It is not difficult to see that there is a general drop of the temperature around frame 20. In fact, I can average over all stations:

ListLinePlot[Mean /@ Transpose[
   Table[Differences@(ToExpression@#[[2, n]] & /@ Delete[netatmoeuropedata, 14])[[All, 3]], {n, 1, 1200}]], 
 Frame -> True, FrameLabel -> {"Frame", "Temperature Difference"}, Epilog -> {{Red, Opacity[0.2], Rectangle[{18, -0.9}, {21, 1}]}}]

and I see again a marked drop in (the increase of the) temperature during the eclipse:

This is more difficult to see than in the case of my own Netatmo station, because (i) my station measure every 5 minutes whereas the public data is only updated every 30 minutes, and (ii) because I live in a place where the solar eclipse was about 94% and when I average over all European stations that includes countries with much lower percentages.

So then there remains the question of the power grid. It turns out that in the UK there is a portal which gives access to a wealth of data from the power grid: http://www.gridwatch.templar.co.uk

You can download data in csv format (see attached to this post "gridwatch-4.csv"). The data is measured in 5 minutes intervals. SemanticImport does the trick:

data = SemanticImport["~/Desktop/gridwatch-4.csv"]

Now we can look at the data. The frequency of the AC should optimally be at around 50Hz. If the system is strained the frequency drops. The system is carefully controlled to maintain that frequency-let's see whether we can find evidence for control. We can plot the frequency during the solar eclipse:

DateListPlot[Lookup[data // Normal, {"timestamp", "frequency"}], 
 Epilog -> {{Red, Opacity[0.2], Rectangle[{DateList[{2015, 3, 20, 8, 33, 0}], 49.87}, {DateList[{2015, 3, 20, 10, 46, 0}], 50.22}]}}, 
 AspectRatio -> 0.25, FrameLabel -> {"Time", "Frequency Hz"}, LabelStyle -> Directive[Bold, Medium], Filling -> Bottom]

So, nothing much there. The frequency was quite constant. Obviously, the system was quite under control in spite of the Photovoltaic power "cut". This is interesting, because I heard that there is a job in a national grid company here, where someone has to watch a popular program "Eastenders"; because when the program finishes, people apparently "put the kettle on", i.e. make tea, and that increases the power consumption. That person then has to add every from other countries, e.g. France, and/or hydroelectric power from Scotland to keep the network stable. Hydroelectric power is very convenient here, because it is easy to "switch on and off". This is different from other sources like nuclear power plants and wind power. So let's have a look at the hydroelectric power production:

DateListPlot[Lookup[data // Normal, {"timestamp", "hydro"}], 
 Epilog -> {{Red, Opacity[0.2], Rectangle[{DateList[{2015, 3, 20, 8, 33, 0}], 0}, {DateList[{2015, 3, 20, 10, 46, 0}], 850}]}}, 
 AspectRatio -> 0.25, FrameLabel -> {"Time", "Engergy MW"}, LabelStyle -> Directive[Bold, Medium], Filling -> Bottom]

Hmm, that looks as if somebody switched something on here. It might perhaps be a sign of active control. I could not find any evidence for a change in the other energy sources such as wind and nuclear. The power demand went a bit up during that time:

DateListPlot[Lookup[data // Normal, {"timestamp", "demand"}], 
 Epilog -> {{Red, Opacity[0.2], Rectangle[{DateList[{2015, 3, 20, 8, 33, 0}], 29000}, {DateList[{2015, 3, 20, 10, 46, 0}], 50000}]}}, 
 AspectRatio -> 0.25, FrameLabel -> {"Time", "Engergy MW"}, LabelStyle -> Directive[Bold, Medium], Filling -> Bottom] 

I am not quite sure why that might be, but perhaps lot of us "put the kettle on" during and after the event!? All these results are of course quite un-scientific, but it there is no evidence that our power grid was at the risk of a major blackout. But then the insolation in Scotland is quite low and clouds are not unheard of either. Perhaps the reliance on solar power is not large enough to impact the system substantially.

There are many more things to try out; we have only used a tiny amount of the available data.

All the best from Aberdeen,

Marco

PS: The netatmo data can be downloaded as a zip file here.

Attachments:

Outdoor_20_3_2015.csv

gridwatch-4.csv