Message Boards

GROUPS:

Chandramauli Agrawal

[WIS22] Neural network to study cosmic air showers

Chandramauli Agrawal, Indian Institute of Science Education and Research Bhopal

Posted 7 months ago

899 Views

0 Replies

3 Total Likes

Follow this post

Neural network to study cosmic air showers
by Chandramauli Agrawal
Indian Institute of Science Education and Research Bhopal

Cosmic Air showers are atmospheric phenomena, which occurs when a cosmic ray hits the upper level of the atmosphere. This initial collision sends a cascade of events downstream into the atmosphere, which can be detected.
The project is inspired from a paper (link in reference) discusses how a air shower characteristics can be found out using a trained network. The project uses a preexisting data, understands it, and train a neural network. The shower property in focus is the maximum penetration of the cosmic ray.

What are Cosmic Air Showers?

An air shower is cascade of ionized particles and EM waves produced when a cosmic ray (produced in space) enters the atmosphere.

Image Source: See references

The cosmic ray particle can be a proton, an electron, nucleus, a photon or sometimes even positron, strikes an atom’s nucleus in the air producing many energetic hadrons, which ultimately decays to produce other particles and EM radiations.

Importing the Dataset in Mathematica

The data for the project is obtained from the following link: https://desycloud.desy.de/index.php/s/YHa79Gx94CbPx8Q/download.
The downloaded file is in NPZ format, i.e. it is a Zip file containing multiple “.npy” files.

The downloaded file named: 4_airshower_100k_regression.npz was first extracted and it was found to have contained 8 “.npy” files namely:

X_train_0.npy

X_train_1.npy

X_test_0.npy

X_test_1.npy

X_train_2.npy

X_test_2.npy

y_train.npy

y_test.npy

Note: All the green highlighted files were the input, blue-the coordinates and red were the ouput to our neural network.
Mathematica does not directly import “.npy” files so we came up with two ways to do it.
Method-2 is superior due to the over-committing memory issues arising with Method-1.

Method-1: Using ExternalEvaluate

As the name suggests ExternalEvaluate function allows evaluation of an expression using some outer environment (non-Mathematica), in this case Python.

Initiating:

In our earlier workings we were facing issues with Python version 3.9.1
To overcome them, we used an old version (Python 3.6.13). First starting with registering the path to our required version environment.

In[]:=

RegisterExternalEvaluator["Python","C:\Users\chand\anaconda3\envs\python\python.exe"]

Out[]=

94fe59e4-e21b-2853-85ae-508c49ac2d4c

In[]:=

FindExternalEvaluators["Python"]

Out[]=

	System	Version	Target	+	Registered
60339dff-7dfe-ed70-ac9b-261eae8ffc3a	Python	3.9.1	C:\Users\chand\AppData\Local\Programs\Python\Python39\python.exe	•	Automatic
94fe59e4-e21b-2853-85ae-508c49ac2d4c	Python	3.6.13	C:\Users\chand\anaconda3\envs\python\python.exe	•	True

In[]:=

py=StartExternalSession[<|"System""Python","Version""3.6.13"|>]

Out[]=

ExternalSessionObject

System: Python	Kernel: 3.6.13
UUID: 90504bcc-6a6b-4fe4-8841-e3089bd3ce77



Input (Training and Testing):

We then started bringing the data on Mathematica.
Considering the quantity of data, we will only deal with limited number of data points. We took,

In[]:=

sizeTrain=700;sizeTest=300;

In[]:=

temp=ExternalEvaluate[py,"import numpy as npnp.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\X_train_0.npy')"];xtrain0=temp[[1;;sizeTrain,All,All,All]];ClearAll[temp];

temp=ExternalEvaluate[py,"import numpy as npnp.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\X_train_1.npy')"]xtrain1=temp[[1;;sizeTrain,All,All,1]];ClearAll[temp];

temp=ExternalEvaluate[py,"np.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\X_test_0.npy')"]xtest0=temp[[1;;sizeTest,All,All,All]];ClearAll[temp];

temp=ExternalEvaluate[py,"np.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\X_test_1.npy')"]xtest1=temp[[1;;sizeTest,All,All,1]];ClearAll[temp];

Detector Coordinates:

In[]:=

xtrain2=ExternalEvaluate["Python","import numpy as np np.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\X_train_2.npy')"]

Out[]=

NumericArray

Type: Real64

Dimensions: {81,3}



In[]:=

xtest2=ExternalEvaluate[py,"import numpy as np np.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\X_test_2.npy')"]

Out[]=

NumericArray

Type: Real64

Dimensions: {81,3}



In the given data, both the coordinates for training and testing were found to be same. We can, hence, essentially only use one for our purposes.

In[]:=

xtrain2xtest2

Out[]=

True

Output (Training and Testing):

temp=ExternalEvaluate[py,"import numpy as npnp.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\y_train.npy')"];ytrain=temp[[1;;sizeTrain]];ClearAll[temp];

temp=ExternalEvaluate[py,"import numpy as npnp.load(r'C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\y_test.npy')"];ytest=temp[[1;;sizeTest]];ClearAll[temp];

Method-2: Through CSV format

Our data is first converted into a “.csv” format, and then brought on Mathematica. Similar to what we did previously, we will deal with only a small set of data-points.
Note: The imported data is a 1D list, and thus ArrayReshape is necessary to convert it into the required arrays.

In[]:=

sizeTrain=70;sizeTest=30;

Input (Training and Testing):

In[]:=

xTrain0=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\xTest0Small.csv","Data"];xTrain0=ArrayReshape[xTrain0,{sizeTrain,9,9,80,1}];xTrain1=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\xTrain1Small.csv","Data"];xTrain1=ArrayReshape[xTrain1,{sizeTrain,9,9,1}];xTest0=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\xTest0Small.csv","Data"];xTest0=ArrayReshape[xTest0,{sizeTest,9,9,80,1}];xTest1=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\xTest1Small.csv","Data"];xTest1=ArrayReshape[xTest1,{sizeTest,9,9,1}];

Detector Coordinates

In[]:=

xTrain2=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\xTrain2.csv","Data"];xTrain2=ArrayReshape[xTrain2,{81,3}];xTest2=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\xTest2.csv","Data"];xTest2=ArrayReshape[xTest2,{81,3}];

Output (Training and Testing):

In[]:=

yTrain=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\yTrainSmall.csv","Data"];yTest=Import["C:\\Users\\chand\\Desktop\\Wolfram Winter School\\Dataset\\Shower_data(small)_csv\\yTestSmall.csv","Data"];

Understanding the Data:

Since the data was given and not produced by us, it was crucial to understand it’s entries.
We know the following information:

◼

xTrain2 and xTest2 are the coordinates of detectors.

In[]:=

Dimensions[xTest2]

Out[]=

{81,3}

These are the entries of the array.

In[]:=

Dataset[xTrain2//Normal]

Out[]=

-6000.0	-6000.0	1400.0
-6000.0	-4500.0	1400.0
-6000.0	-3000.0	1400.0
-6000.0	-1500.0	1400.0
-6000.0	0.0	1400.0
-6000.0	1500.0	1400.0
-6000.0	3000.0	1400.0
-6000.0	4500.0	1400.0
-6000.0	6000.0	1400.0
-4500.0	-6000.0	1400.0
-4500.0	-4500.0	1400.0
-4500.0	-3000.0	1400.0
-4500.0	-1500.0	1400.0
-4500.0	0.0	1400.0
-4500.0	1500.0	1400.0
-4500.0	3000.0	1400.0
-4500.0	4500.0	1400.0
-4500.0	6000.0	1400.0
-3000.0	-6000.0	1400.0
-3000.0	-4500.0	1400.0
rows 1–20 of 81

, and how they would look on a plane at the z-coordinate: 1400 (m)

ListPlot[xTest2[[All,1;;2]],PlotRangeAll]

Out[]=

Note that the length scale used here is Meters.

We find that both the arrays are essentially equal.

In[]:=

xTrain2==xTest2

Out[]=

True

Let’s also check the dimension to make sure we are on the right track!

◼

xTrain0 and xTest0 contains the signal bins distributed over 80 time bins, with each bin of size 25 ns, for a given detector and a particular event.

In[]:=

Dimensions[xTrain0]

Out[]=

{70,9,9,80,1}

In[]:=

Dimensions[xTest0]

Out[]=

{30,9,9,80,1}

In[]:=

eventNumber=10;

In[]:=

Manipulate[Dataset[Normal[xTrain0[[eventNumber,All,All,timeBin,1]]]],{timeBin,1,80,1},SaveDefinitionsTrue]

Out[]=

timeBin

-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	1.077283859252929688	2.095056772232055664	2.61784815788269043	-1.0	-1.0	-1.0	-1.0
-1.0	0.06422217190265655518	3.1916046142578125	13.28003406524658203	51.97373199462890625	5.418806552886962891	0.2873561680316925049	-1.0	-1.0
-1.0	-1.0	1.14610743522644043	4.022394180297851562	7.516362190246582031	2.449770450592041016	1.088339686393737793	-1.0	-1.0
-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0

Signals at a detector adjacent to the central detector.

In[]:=

ListLinePlot[Flatten[Table[xTrain0[[eventNumber,4,5,n]],{n,1,80}]],AxesLabel{"time","signal"},PlotRangeAll,PlotLabel"Signals on Detector:(3,5)"]

Out[]=

Comparing the above values with the readings obtained on the central detector.

In[]:=

ListLinePlot[{Flatten[Table[xTrain0[[eventNumber,5,5,n]],{n,1,80}]],Flatten[Table[xTrain0[[eventNumber,3,5,n]],{n,1,80}]]},AxesLabel{"time","signal"},PlotRangeAll,PlotLegends{"Detector:(5,5)","Detector:(3,5)"}]

Out[]=

	Detector:(5,5)
	Detector:(3,5)

The signal strength in the other detector is subdued by that from the central detector(where our cosmic shower was centered).

◼

xTrain1 and xTest0 contains the knowledge of first modified arrival time

In[]:=

Dimensions[xTrain1]

Out[]=

{70,9,9,1}

In[]:=

Dimensions[xTest1]

Out[]=

{30,9,9,1}

In[]:=

Total[Table[Ramp[xTrain0[[eventNumber,All,All,i]]],{i,1,80}],1];

In[]:=

Manipulate[Dataset[xTest1[[event,All,All,1]]],{event,1,30,1},SaveDefinitionsTrue]

Out[]=

event

-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	-1.0	0.00001222711216541938484	-1.0	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	-1.0	0.000007769791409373283386	0.000007363518761849263683	-1.0	-1.0	-1.0	-1.0
-1.0	-1.0	-1.0	0.000003696791736729210243	0.000003156207867505145259	-1.0	-1.0