Paid PractitionerCertificate

Cleaning the Signal

Outlier detection and smoothing taught as one subject, because they are the two ways to be wrong about the same record: how to find the values that should not be trusted, how to quieten a noisy channel without erasing the event you were looking for, and how to defend both choices to someone who disagrees.

Lessons
8
Quiz questions
65
Exercises
1 (10 min)
Calculators
1
Time
About 86 min
Price
€179+ VAT
Certificate
Graded final exam, then a PDF certificate with a number anyone can verify
Who it's for
Performance analysts and engineers who build or audit the step between raw shipboard records and the numbers a report quotes: anyone who has ever set a filter threshold and been unable to say where the number came from.
Assumed knowledge
Our course Describing Vessel Data, or equivalent comfort with distributions, spread and the normality question. Every method here assumes something about the distribution, and the previous course is how you check it.
Taught by

Christos Papandreou

CEO & Founder, Blue Autonomy

  • Naval architect and marine engineer, National Technical University of Athens
  • 15 years of experience in fleet performance and operational data
  • Chief Scientific Officer, NEMO, previously DeepSea Technologies, StormGeo, Athenian Sea Carriers, PwC

Part of the Performance Data Science specialisation.

What you will be able to do

By the end of the course. Each outcome names the lessons that deliver it, and the final exam samples every one of them.

  1. Sort suspect records into impossible, improbable and interesting, and design a cleaning pipeline that flags rather than deletes, reports what it removed, and can be run again. Lesson 1
  2. Choose and apply a univariate outlier test on the evidence of the distribution (the z-score, the IQR rule, the robust MAD z-score or Chauvenet's sample-size-aware criterion), stating what each assumes and why masking defeats the z-score exactly when it is most needed. Lessons 2, 3
  3. Detect steady-state operation using windowed rejection followed by block acceptance. Lesson 3
  4. Detect records that are unremarkable in each variable and wrong in combination, using regression residuals, an errors-in-variables fit, the binned-band method and the Mahalanobis distance with its chi-square cut-offs, and state the assumptions that make each unsafe on raw speed-power data. Lessons 4, 5
  5. Apply moving-average, median, Savitzky-Golay and Kalman filters, matching the filter to the job (removing a spike, keeping a peak, following a trend), fixing the order in which cleaning stages run, and quantifying the improvement as signal-to-noise. Lessons 6, 7
  6. Defend a smoothing window with variance reduction and RMSE together, recognising that neither has an optimum on its own, budgeting the lag it introduces and setting the window from the timescale of the phenomenon being preserved. Lesson 8

Cleaning data is where analysis quietly becomes advocacy. Every threshold is a decision about which records count, and a chain of small, undocumented decisions can move a speed-loss figure by more than the effect being measured. The purpose of this course is to make each of those decisions explicit, principled and reversible.

What is inside: a taxonomy that separates the physically impossible record from the merely unusual one, and treats them differently; the univariate outlier tests, z-score, interquartile range and the median absolute deviation, with the assumptions each one makes; Chauvenet's criterion and the window that turns it from a blunt instrument into a steady-state detector; bivariate detection through regression residuals, binned bands and the Mahalanobis distance, which finds points that are unremarkable in either variable and rare as a pair; the smoothing family from the simple moving average through weighted and exponential forms to the median filter, Savitzky-Golay and the Kalman filter; and the arithmetic of choosing a window, so that the number is defended by variance reduction and RMSE rather than by taste.

The opening lesson is free to read as a sample. By the end you should be able to write a cleaning pipeline for a fleet, state what each stage assumes, quantify what it removed, and answer the only question that matters when a charterer disputes your figure: what would the answer have been if you had made the other choice?

What the course covers

  1. Impossible, Improbable, Interesting 11 min
  2. Univariate Outliers: Z-score, IQR and MAD 19 min + exercise
  3. Chauvenet's Criterion, and the Window That Makes It Work 9 min
  4. Two Variables at Once: Residuals and Binned Bands 10 min
  5. Mahalanobis Distance: Outliers in the Shape of the Cloud 8 min
  6. Smoothing I: Moving Averages, Weights and the Median Filter 11 min
  7. Smoothing II: Savitzky-Golay, Kalman and the Order of Operations 9 min
  8. Choosing the Window: Variance Reduction, RMSE and Lag 9 min

€179 + VAT

€221.96 including 24% VAT, which is what an individual in the EU pays. Outside the EU no VAT is added. A company in another EU country pays the net price once its VAT number is confirmed in VIES, the EU's register; a company in Greece pays the 24%. Checkout shows the total before you pay.

Paid once, for one person, for as long as the Academy runs: every lesson, the Excel workbook, the final examination and the certificate. Part of the Performance Data Science specialisation, which costs less than its courses bought one at a time.

Buying online needs JavaScript switched on in your browser. Otherwise write to info@blueautonomy.gr and we arrange access.

All prices · Refunds: 7 days, unless you have started

Common questions

What exactly is included?

8 lessons (about 86 minutes in total), 65 quiz questions with explained answers, and 1 working Excel calculator: Univariate outlier tests.

How much does it cost?

€179 plus VAT, paid once: €221.96 with 24% VAT, which is what an individual in the EU pays; outside the EU no VAT is added, and a company in another EU country with a VAT number valid in VIES pays the net price. That buys every lesson, the Excel workbook, the final examination and the certificate, for one person, for as long as the Academy runs. The course is also part of the Performance Data Science specialisation, which costs less than its courses bought one at a time. Everything else is on the pricing page.

Is there work to do, or is it only reading?

1 lesson carries a required exercise, about 10 minutes of hands-on work in total. You download that lesson's Excel workbook, follow the numbered steps against the numbers the lesson states, then change one input and watch the answer move. The exercises are where the method stops being something you have read and becomes something you can run; the stated course time includes them.

Do I get a certificate?

Yes. The course ends with a graded final exam, marked on our server rather than in your browser: you see your score and which questions you got wrong, but not the answers, because the questions are drawn from a pool. Pass it and you get a PDF certificate in your name, with a certificate number on it.

That number is the point. Anyone you show the certificate to can check it themselves at academy.blueautonomy.gr/verify, with no account and no email to us: the page states the name, the course and the date we issued it. A certificate nobody can verify is a decorated PDF, and we would rather issue one that stands up.

Can I preview the course?

Yes: the first lesson, Impossible, Improbable, Interesting, is free to read in full. It is a real lesson, not a trailer: same depth, same quiz.

Do I need prior knowledge?

Our course Describing Vessel Data, or equivalent comfort with distributions, spread and the normality question. Every method here assumes something about the distribution, and the previous course is how you check it.