Skip to content

Batch missing-data handling and smoothing filters #200

@kgdunn

Description

@kgdunn

Migrated from TODO.md (Batch section).

Items covered

  • Missing-data handling in batch data. (TODO.txt)
  • Smoothing tools: lowess and Savitzky-Golay filter for batch data. (TODO.txt)

Approach

Replace the crude bfill/ffill approach with a principled fill, and add lowess + Savitzky-Golay smoothing helpers.

The current crude approach, preserved verbatim from the original TODO.txt:

# Fill in missing values
def fill_na_values(df):
    "Very very crude method for now"
    return df.fillna(method='bfill').fillna(method='ffill')

missing_filled = {}
for batch_id, batch in  df_dict.items():
    missing_filled[batch_id] = fill_na_values(batch)

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions