DASCore

The Fundamentals

DASDAE developers

DASCore: Data Structures


Patch

Spool

DASCore: Patch


Patch: Contiguous data and metadata

  • Contains three types of metadata
    • dims - dimension labels
    • coords - dimension values
    • attrs - scalar metadata
  • Maintains metadata consistency
  • Strives to be immutable

Patch design inspired by Xarray

DASCore: Patch


  • We start by importing dascore
import dascore as dc

DASCore: Patch


  • Then we read a DAS file
import dascore as dc

patch = dc.spool('path_to_das_file.h5')[0]

DASCore: Patch


  • Or we can use one of the example patches
import dascore as dc

patch = dc.get_example_patch("example_event_1")

DASCore: Patch


  • Simple visualizations are accessed through .viz namespace
import dascore as dc
patch = dc.get_example_patch("example_event_1")

patch.viz.waterfall(show=True, scale=0.1)

DASCore: Patch

Stanvek et al., 2022

DASCore: Patch

  • Processing methods are chained together
import dascore as dc
patch = dc.get_example_patch("example_event_1")

patch_proc = (
    patch.set_units("1/s", distance="m", time="s")
    .detrend("time")
    .taper(time=0.05)
    .pass_filter(time=(..., 300))
)

patch_proc.viz.waterfall(show=True, scale=0.35)

DASCore: Patch

DASCore: Patch


DASCore Patch ⚡
---------------
➤ Coordinates (distance: 300, time: 2000)
    *distance: CoordRange( min: 0 max: 299 step: 1 shape: (300,) dtype: int64 units: m )
    *time: CoordRange( min: 2017-09-18 max: 2017-09-18T00:00:07.996 step: 0.004s shape: (2000,) dtype: 
datetime64[ns] units: s )
➤ Data (float64)
   [[0.778 0.238 0.824 ... 0.37  0.077 0.232]
    [0.497 0.442 0.703 ... 0.126 0.118 0.78 ]
    [0.207 0.195 0.174 ... 0.849 0.365 0.807]
    ...
    [0.619 0.105 0.669 ... 0.621 0.436 0.5  ]
    [0.757 0.259 0.091 ... 0.361 0.937 0.104]
    [0.158 0.295 0.585 ... 0.229 0.24  0.494]]
➤ Attributes
    tag: random
    category: DAS

DASCore: Patch


  • Updating metadata
import dascore as dc
patch = dc.get_example_patch()

new = patch.update_attrs(tag='perf 2')

DASCore: Patch


  • Updating coordinates
import dascore as dc
patch = dc.get_example_patch()

time = '2017-09-18T10:00:01'
new = patch.update_coords(time_min=time)

DASCore: Patch


  • Escape hatches: ObsPy
import dascore as dc
patch = dc.get_example_patch()

stream = patch.io.to_obspy()

DASCore: Patch


  • Escape hatches: xarray DataArray
import dascore as dc
patch = dc.get_example_patch()

data_array = patch.io.to_xarray()

DASCore: Patch


  • Units
import dascore as dc
patch = dc.get_example_patch()

patch_units = patch.convert_units(
  "m/s", 
  distance="ft",
)

DASCore: Patch


  • Units
import dascore as dc
from dascore.units import Hz, s

patch = dc.get_example_patch()

patch_filt = patch.pass_filter(
  time=(10*Hz, 40*Hz)
)

DASCore: Patch


  • Units
import dascore as dc
from dascore.units import Hz, s

patch = dc.get_example_patch()

patch_filt = patch.pass_filter(
  time=(1*s, 2*s)
)

DASCore: Spool


Spool: Collection of Patches

  • Provides access to data sources
    • In-memory (list of patches)
    • On-disk (directory of files)
    • Remote (cloud resources)*
  • Orchestrates batch processing
  • Operates lazily

*not yet implemented

DASCore: Spool


  • Getting a spool from: A single file
import dascore as dc

spool = dc.spool('path_to_das_file.h5')

DASCore: Spool


  • Getting a spool from: A directory of DAS files
import dascore as dc

spool = dc.spool('das_directory').update()

DASCore: Spool


  • Getting a spool from: DASCore’s example data set
import dascore as dc

spool = dc.get_example_spool()

DASCore: Spool


  • Patches are accessed via indexing
import dascore as dc
spool = dc.get_example_spool()

patch = spool[0]

DASCore: Spool


  • or via iteration
import dascore as dc
spool = dc.get_example_spool()

for patch in spool:
  ...

DASCore: Spool


  • Sub-spools can be created with slices
import dascore as dc
spool = dc.get_example_spool()

sub_spool = spool[:2]

DASCore: Spool


  • spools are filtered with select
import dascore as dc
spool = dc.get_example_spool()

subset_spool = spool.select(
  tag='experiment1',
  time=(None, '2021-01-01'),
  distance=(10, 100)
)

DASCore: Spool


  • Data are chunked with chunk
import dascore as dc
spool = dc.get_example_spool()

chunked_spool = spool.chunk(
  time=60,
  overlap=10,
)

DASCore: Spool


  • chunk can also merge adjacent/overlapping data
import dascore as dc
spool = dc.get_example_spool()

chunked_spool = spool.chunk(
  time=None,
)