Message Boards

GROUPS:

Staff Picks Data Science Image Processing Audio, Sound and Music Mathematica Graphics and Visualization Wolfram Language Modeling Machine Learning Wolfram High School Summer Camp

Wolfram Emerging Leaders Program

[WELP20] Acoustic Modeling with Generated Depth Maps

Wolfram Emerging Leaders Program, Wolfram Summer Programs

Posted 7 months ago

2185 Views

0 Replies

10 Total Likes

Follow this post

The Wolfram Emerging Leaders Program is a 4 month long project based program designed for gifted high school students to take a deep dive into a topic of their choice. Students work remotely in small groups to take their project from ideation to completion, with the guidance of Wolfram experts. Wolfram Emerging Leaders Program participants are selected from the Wolfram High School Summer Camp, which is open to talented, STEM oriented students age 17 and under.
This essay was written by:

◼

Timothy Bilik

◼

Logan Gilbert

◼

Emily Hyun

◼

Shubham Kumar

Introduction

This project aims to explore the propagation of sound in three dimensional regions. We begin by experimenting with acoustic modeling using primitive shapes.
After coming up with a working acoustic model, we utilized a depth sensing neural network to extrapolate data from single images. Using this data, we were able to create a boolean region that is representative of the environment being surveyed.
We integrated this depth map with our acoustic model to obtain attenuation coefficients which were applied to sample audio.
Throughout the process, we were able to come across interesting visualizations, learn about the basic physics behind acoustics and explore digital signal processing.

2D Modeling

The Helmholtz Equation

This partial differential equation is a time independent form of the Wave Equation and is used in physics to model many phenomena. We use it hear to model the pressure distribution in a region when different frequencies are emitted from a monopole sound source.

Preliminary Functions & Attributes from PDE Models Documentation

In[]:=

MaxGridSize[c_,ω_,Resolution_]:=Module[{λ},λ=2 π c/ω;λ/Resolution]Options[AcousticsOptions]={"Resolution"12};AcousticsOptions[c_,ω_,opts:OptionsPattern[]]:=Sequence[Method{"PDEDiscretization"{"FiniteElement","MeshOptions"{"MaxCellMeasure"{"Length"MaxGridSize[c,ω,OptionValue["Resolution"]]}}}}]RegularizedDelta[γ_,X_List,Xs_List]:=Piecewise[{{Times@@Thread[1/(4 γ) (1+Cos[π/(2 γ) (X-Xs)])],And@@Thread[RealAbs[X-Xs]≤2 γ]},{0,True}}]

air

=QuantityMagnitude[ThermodynamicData["Air","SoundSpeed"]];

air

=QuantityMagnitude[ThermodynamicData["Air","Density"]];h[ω_] := MaxGridSize[

air

, ω, 12]



=1;Q[ω_, X_List, Xs_List] := RegularizedDelta[h[ω]/2, X, Xs]*



; (*ClearAll[AcousticModel]*)AcousticModel[p_,X_List,ρ_,{Q_,F_}]:=Module[{a,factor},factor=-1/ρ*IdentityMatrix[Length[X]];a=PiecewiseExpand[Piecewise[{{factor,True}}]];Inactive[Div][a.Inactive[Grad][p,X],X]-Inactive[Div][F/ρ,X]-Q]

Creating The Simulation Domain

In[]:=

r=1;Ω=Rectangle[{0,0},{r,r}];RegionPlot[Ω]

Out[]=

Setting Sound Source Location

In[]:=

xs=0.5;ys=0.5;r=1;

Creating the Acoustic Model

In[]:=

monopoleModel2D=AcousticModel[p[x,y],{x,y},ρ,{Q[ω,{x,y},{xs,ys}],{0}}]-

p[x,y]

Out[]=

p[x,y]

7.73542× -6 10 2 ω (1+Cos[0.0174752(-0.5+x)ω])(1+Cos[0.0174752(-0.5+y)ω])	RealAbs[-0.5+x]≤ 179.774 ω &&RealAbs[-0.5+y]≤ 179.774 ω
0	True

∇

{x,y}

·-

,0,0,-

.

∇

{x,y}

p[x,y]-

∇

{x,y}

·{0}

Setting a Boundary Condition

In[]:=

abcf

=NeumannValue-

ω

p[x,y]

,True;

Creating the PDE

In[]:=

pde={monopoleModel2D

abcf

}/.{c

air

,ρ

air

};

Solving the PDE

In[]:=

pfunMonopole2D=ParametricNDSolveValue[pde,p,{x,y}∈Ω,{ω},AcousticsOptions[

air

,ω]]

Out[]=

ParametricFunction



Expression: p

Parameters: {ω}



Visualizing the Results

In[]:=

pMonopole500Hz=pfunMonopole2D[2πf/.{f500}]

InterpolatingFunction

Domain: {{0.,1.},{0.,1.}}

Output: scalar



Some examples

In[]:=

;

These are plots of the pressure distributions of various regions that we simulated sound propagation in. Each exhibited different characteristics and provided insight on how we can tune our model.

3D Modeling with Primitive Shapes

The Helmholtz Equation

Creating the Simulation Domain

In[]:=

Ω=Region[Cone[{{0,0,0},{0,0,2}},1]];RegionPlot3D[Ω]

Out[]=

Setting Sound Source Location

In[]:=

xs=0;ys=0;zs=0;r=1;

Creating the Acoustic Model

In[]:=

monopoleModel3D=AcousticModel[p[x,y,z],{x,y,z},ρ,{Q[ω,{x,y,z},{xs,ys,zs}],{0}}]-

p[x,y,z]

;

Setting a Boundary Condition

In[]:=

abc

=NeumannValue-

ω

p[x,y,z]

,True;

Creating the PDE

pde={monopoleModel3D

abc

}/.{c

air

,ρ

air

};

Solving the PDE

pfunMonopole3D=ParametricNDSolveValue[pde,p,{x,y,z}∈omega,{ω},AcousticsOptions[

air

,ω]]

Visualizing Results

pMonopole500HzCone=pfunMonopole3D[2πf/.{f500}];

Examples & Tests

While we tried to generate visualizations for this, the process was too computationally intensive. In the future, we will attempt to render high quality plots to further investigate this model.

Depth Map

Single-Image Depth Perception Net

Here is an example image from Wolfram:

In[]:=

img=

;

A neural network exists that can convert an image into a depth map that doesn’t require multi-view reconstruction.

We obtain the depth map using the neural network “Single-Image Depth Perception Net Trained on NYU Depth V2 and Depth in the Wild Data”.

In[]:=

depthMap=NetModel["Single-Image Depth Perception Net Trained on NYU Depth V2 and Depth in the Wild Data"][img];

This allows us to view the depth in a two dimensional visualization

In[]:=

ImageAdjust[Image[depthMap]]

Out[]=

We can better visualize the depthMap with ListPlot3D, which should give us a 3D rendering of the depth of the image.

In[]:=

ListPlot3D[depthMap,PlotStyleTexture[img]]

Out[]=

Notice that some of the depth data has been inaccurately captured. The distance between the farthest point and the closest point in the image doesn’t appear to be recreated in the depth map. The accuracy of relative positions between the points isn’t maintained.

Fine Tuning the output

We found that multiplying the created depth map by the square root of itself can sometimes yield a more accurate depth map. An example is listed below.

Sample Image:

In[]:=

img=

;

Again, we obtain the depthMap using the neural net.

In[]:=

depthMapModel=NetModel["Single-Image Depth Perception Net Trained on NYU Depth V2 and Depth in the Wild Data"][img];

This time, we apply a square root function to the depth values.

In[]:=

depthMapModel=Sqrt[Abs[Reverse@depthMapModel]]*(-Reverse@depthMapModel);

Visualizing the plot gives us a smooth and accurate depth perception map.

In[]:=

ListPlot3D[depthMapModel,PlotStyleTexture[img],PlotTheme{"Minimal","NoAxes"},ViewPoint{0,-0.96,1.84}]

Out[]=

We’ve incorporated some post-processing to help improve the accuracy of depth perception here.

Creating a Boolean Region

Method 1 - Hexahedrons

A depth map can be re-created into a boolean region by converting each depth map point into a hexahedron, where the depth value is associated with the height of each hexahedron.

Obtaining a depth map:

In[]:=

depthMap=NetModel["Single-Image Depth Perception Net Trained on NYU Depth V2 and Depth in the Wild Data"]

;

Creating the vertices defining each hexahedron that will create the region:

In[]:=

pts=Table[{{x-1,y-1,-50},{x-1,y,-50},{x,y,-50},{x,y-1,-50},{x-1,y-1,depthMap[[x]][[y]]*10},{x-1,y,depthMap[[x]][[y]]*10},{x,y,depthMap[[x]][[y]]*10},{x,y-1,depthMap[[x]][[y]]*10}},{x,1,Dimensions[depthMap][[1]],1},{y,1,Dimensions[depthMap][[2]],1}];

In[]:=

pts=Flatten[pts,2];

Creating a Mesh Region with the hexahedrons:

In[]:=

Ω=MeshRegion[pts,Hexahedron[Partition[Range[240*320*8],8]]]

Out[]=

Using this mesh region we were able to accurately translate the depth data from an image to a boolean region. Note that each pixel in the image has a depth value associated with it, which dictated how tall the hexahedron corresponding to that pixel is. We simply combined all of these hexahedrons to create the region shown above. We need our data in this format to be compatible with our acoustic model. In order for the mesh region to be compatible, it needs to fit in a 1 by 1 by 1 region. The function below accomplishes that. We simply rescale the values to fit this constraint.

In[]:=

Image2MeshRegion[image_Image,minBound_Integer:0,maxBound_Integer:1]:=Module[{depthMap,minDepth,maxDepth,inc, width, height, pts},(If[maxBound≤minBound,Return[]];depthMap = NetModel["Single-Image Depth Perception Net Trained on NYU Depth V2 and Depth in the Wild Data"][image];minDepth = Min[depthMap];maxDepth = Max[depthMap];depthMap = Rescale[depthMap,{minDepth,maxDepth},{minBound+0.001,maxBound}]; (*the 0.001 is in there to prevent degenerate hexahedrons*)width = Dimensions[depthMap][[1]];height = Dimensions[depthMap][[2]];inc = Divide[maxBound-minBound+0.0,Max[{width,height}]];pts = Table[{{(x-1)*inc,(y-1)*inc,minBound},{(x-1)*inc,y*inc,minBound},{x*inc,y*inc,minBound},{x*inc,(y-1)*inc,minBound},{(x-1)*inc, (y-1)*inc,depthMap[[x]][[y]]},{(x-1)*inc,y*inc,depthMap[[x]][[y]]},{x*inc,y*inc,depthMap[[x]][[y]]},{x*inc,(y-1)*inc,depthMap[[x]][[y]]}}, {x, width}, {y, height}];MeshRegion[Flatten[pts,2],Hexahedron[Partition[Range[width*height*8],8]]])]

Method 2 - Discretized Region Plot

The depth map can be interpolated into a function and then discretized as shown below.

Obtaining an Interpolation Function for the data:

In[]:=

f=ListInterpolation[Rescale[depthMap,{Min[depthMap],Max[depthMap]},{0.0001,1}],{{0,1},{0,1}},InterpolationOrder{10,10}];

Plotting the Region:

In[]:=

rp=RegionPlot3D[z≤f[x,y],{x,0,1},{y,0,1},{z,0,1},PlotPoints100];

Discretizing the region plot:

In[]:=

DiscretizeGraphics[rp]

Out[]=