Salmon in the Loop

A consultant working in an environmental science subdomain focused on counting fish passing through large hydroelectric dams has described the process of building human-in-the-loop machine learning systems for that task, according to an account published by The Gradient.

Large hydroelectric dams are regulated through the Federal Energy Regulatory Commission (FERC), which issues licenses and permits for construction and operation and can enforce compliance through sanctions, fines, or lease termination. To demonstrate compliance, dams are required to routinely produce data showing their operations do not interfere with endangered fish populations in aggregate, typically through fish passage studies built on a primary fish count.

Fish counts can be done visually, with a trained observer incrementing a count as fish move upstream and adding classifications such as illness or injury and hatchery-origin versus wild status. The account notes these attributes may be visible only briefly, and that the job is physically demanding, performed in remote locations under poor lighting and unregulated temperatures. Recording methods vary: some counts are documented with pen and paper, which can introduce transcription error, and dam operators collect data at differing granularity and seasonality.

Some organizations are exploring computer vision and machine learning to automate fish counting, creating a human-in-the-loop system that combines fish biologists' judgment with algorithmic consistency. The described build process has five steps: define the problem space; establish performance goals; collect and label training data; select a model; and monitor system performance, with the loop repeating for further refinement. Most hydropower utilities are interested in solutions meeting a 95% accuracy threshold compared with a regular human visual count, though the account says whether and when that is achievable is heavily negotiated.

Obstacles cited include dependence on expert knowledge and its potential errors and biases; overrepresentation of species such as American shad, which can number hundreds of thousands during a migratory period and obscure Chinook salmon; inadequate illumination and poor image quality; reduced water clarity after seasonal snowmelt; inconsistent data taxonomies across organizations; and model drift. Fish passage studies often monitor relatively few fish, and datasets tied to one season may not represent the full range of species and morphologies, the account states. It also notes background labor issues, describing fish counting as a cost center operators would like to reduce or eliminate.