DASCore

A Python Library for Distributed Acoustic Sensing

DASDAE developers

DASDAE


Distributed Acoustic Sensing Data Analysis Ecosystem

DASDAE


DASCore

  • Foundation for future DASDAE codes
  • Processing routines
  • Basic, customizable visualizations
  • Read/write common data formats
  • Data set management

DASCore: Non-Goals


What DASCore is not:

  • Application for a particular use case
  • Graphical User Interface
  • Machine-learning library
  • Massively parallel framework

DASCore: Installation



conda

conda install dascore -c conda-forge


pip

pip install dascore

DASCore: Data Structures


Patch

Spool

DASCore: Patch


Patch: Contiguous data and metadata

  • Contains three types of metadata
    • dims - dimension labels
    • coords - dimension values
    • attrs - scalar metadata
  • Maintains metadata consistency
  • Strives to be immutable

Patch design inspired by Xarray

DASCore: Patch


  • We start by importing dascore
import dascore as dc

DASCore: Patch


  • Then we read a DAS file
import dascore as dc

patch = dc.spool('path_to_das_file.h5')[0]

DASCore: Patch


  • Or we can use one of the example patches
import dascore as dc

patch = dc.get_example_patch("example_event_1")

DASCore: Patch


  • Simple visualizations are accessed through .viz namespace
import dascore as dc
patch = dc.get_example_patch("example_event_1")

patch.viz.waterfall(show=True, scale=0.1)

DASCore: Patch

Stanvek et al., 2022

DASCore: Patch

  • Processing methods are chained together
import dascore as dc
patch = dc.get_example_patch("example_event_1")

patch_proc = (
    patch.set_units("1/s", distance="m", time="s")
    .detrend("time")
    .taper(time=0.05)
    .pass_filter(time=(..., 300))
)

patch_proc.viz.waterfall(show=True, scale=0.35)

DASCore: Patch

DASCore: Patch


DASCore Patch ⚡
---------------
➤ Coordinates (distance: 300, time: 2000)
    *distance: CoordRange( min: 0 max: 299 step: 1 shape: (300,) dtype: int64 units: m )
    *time: CoordRange( min: 2017-09-18 max: 2017-09-18T00:00:07.996 step: 0.004s shape: (2000,) dtype: 
datetime64[ns] units: s )
➤ Data (float64)
   [[0.778 0.238 0.824 ... 0.37  0.077 0.232]
    [0.497 0.442 0.703 ... 0.126 0.118 0.78 ]
    [0.207 0.195 0.174 ... 0.849 0.365 0.807]
    ...
    [0.619 0.105 0.669 ... 0.621 0.436 0.5  ]
    [0.757 0.259 0.091 ... 0.361 0.937 0.104]
    [0.158 0.295 0.585 ... 0.229 0.24  0.494]]
➤ Attributes
    tag: random
    category: DAS

DASCore: Patch


  • Updating metadata
import dascore as dc
patch = dc.get_example_patch()

new = patch.update_attrs(tag='perf 2')

DASCore: Patch


  • Updating coordinates
import dascore as dc
patch = dc.get_example_patch()

time = '2017-09-18T10:00:01'
new = patch.update_coords(time_min=time)

DASCore: Patch


  • Escape hatches: ObsPy
import dascore as dc
patch = dc.get_example_patch()

stream = patch.io.to_obspy()

DASCore: Patch


  • Escape hatches: xarray DataArray
import dascore as dc
patch = dc.get_example_patch()

data_array = patch.io.to_xarray()

DASCore: Patch


  • Units
import dascore as dc
patch = dc.get_example_patch()

patch_units = patch.convert_units(
  "m/s", 
  distance="ft",
)

DASCore: Patch


  • Units
import dascore as dc
from dascore.units import Hz, s

patch = dc.get_example_patch()

patch_filt = patch.pass_filter(
  time=(10*Hz, 40*Hz)
)

DASCore: Patch


  • Units
import dascore as dc
from dascore.units import Hz, s

patch = dc.get_example_patch()

patch_filt = patch.pass_filter(
  time=(1*s, 2*s)
)

DASCore: Spool


Spool: Collection of Patches

  • Provides access to data sources
    • In-memory (list of patches)
    • On-disk (directory of files)
    • Remote (cloud resources)*
  • Orchestrates batch processing
  • Operates lazily

*not yet implemented

DASCore: Spool


  • Getting a spool from: A single file
import dascore as dc

spool = dc.spool('path_to_das_file.h5')

DASCore: Spool


  • Getting a spool from: A collection of patches
import dascore as dc

patch_list = [dc.get_example_patch()]
spool = dc.spool(patch_list)

DASCore: Spool


  • Getting a spool from: A directory of DAS files
import dascore as dc

spool = dc.spool('das_directory').update()

DASCore: Spool


  • Getting a spool from: DASCore’s example data set
import dascore as dc

spool = dc.get_example_spool()

DASCore: Spool


  • Patches are accessed via indexing
import dascore as dc
spool = dc.get_example_spool()

patch = spool[0]

DASCore: Spool


  • or via iteration
import dascore as dc
spool = dc.get_example_spool()

for patch in spool:
  ...

DASCore: Spool


  • Sub-spools can be created with slices
import dascore as dc
spool = dc.get_example_spool()

sub_spool = spool[:2]

DASCore: Spool


  • spools are filtered with select
import dascore as dc
spool = dc.get_example_spool()

subset_spool = spool.select(
  tag='experiment1',
  time=(None, '2021-01-01'),
  distance=(10, 100)
)

DASCore: Spool


  • Data are chunked with chunk
import dascore as dc
spool = dc.get_example_spool()

chunked_spool = spool.chunk(
  time=60,
  overlap=10,
)

DASCore: Spool


  • chunk can also merge adjacent/overlapping data
import dascore as dc
spool = dc.get_example_spool()

chunked_spool = spool.chunk(
  time=None,
)

DASCore: Spool

  • Parallel Processing
from concurrent.futures import ProcessPoolExecutor
import dascore as dc

def func(patch):
    ...

spool = dc.get_example_spool("random_das")
executor = ProcessPoolExecutor()

out = spool.map(func, client=executor)

DASCore: Supported Formats


  • Terra15
  • TDMS
  • WAV
  • DASDAE
  • Sentek
  • Febus
  • SEGY
  • ProdML
  • DASDAE
  • OptoDAS
  • DASHDF5
  • Pickle

DASCore: Future Work


  • Remote data spool
  • Additional file formats
  • More processing and transformation functions
  • Performance improvements
  • Other DASDAE packages

Acknowledgements

  • Colorado School of Mines ITS and CIARC

  • NSF Geoinformatics grant 2148614

  • DOE STTR grant DE-SC0022478, subcontract 7026-DOE-1T/MINES with Luna Innovations

  • NIOSH funding for Derrick Chambers PhD studies

  • AFRL Acknowledgement: “This material is based on research sponsored by Air Force Research Laboratory (AFRL) under agreement number FA9453-21-2-0018. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.”

  • AFRL Disclaimer: “The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Air Force Research Laboratory (AFRL) or the U.S. Government.”

DASCore: Final Note


  • DASCore is new, there will be:
    • Bugs
    • Missing features
  • DASCore can save you time
  • You can help shape its future!

dascore.org

https://github.com/dasdae/dascore