<!-- greptile_comment -->

<h3>Greptile Summary</h3>


This PR fixes a critical data integrity issue in the NetFlow database processing pipeline. When new files arrive for previously-processed days, the system now detects "stale days" and resets them for complete reprocessing to ensure accurate aggregate statistics.

**Key Changes:**
- Added `get_stale_days()` to detect days with mixed processed/unprocessed files using SQL aggregation
- Added `reset_day_for_reprocessing()` to clear stats and reset file status for stale days
- Added `handle_stale_days()` wrapper called before processing in all 5 stats modules
- Minor improvement: refactored IP parsing in `spectrum_db.py` and `structure_db.py` to avoid duplicate `.strip()` calls
- Minor improvement: added `.strip()` to IP parsing in `ip_db.py` to handle potential whitespace

**Implementation Quality:**
- Clean SQL with proper CTEs for readability
- Handles both `netflow_stats` (uses `timestamp` column) and other stats tables (use `bucket_start` column)
- Proper validation and error messages
- Consistent integration across all processing modules

<h3>Confidence Score: 5/5</h3>


- This PR is safe to merge - it solves a data correctness bug with well-structured code
- The implementation is sound with proper SQL queries, validation, and error handling. The stale day detection logic correctly identifies days needing reprocessing. All five stats processing modules are consistently updated. The minor IP parsing improvements reduce redundant operations without changing functionality.
- No files require special attention

<h3>Important Files Changed</h3>




| Filename | Overview |
|----------|----------|
| netflow-db/discovery.py | Added three functions to handle stale day detection and reprocessing: `get_stale_days`, `reset_day_for_reprocessing`, and `handle_stale_days`. Implementation is clean with proper SQL queries and error handling. |
| netflow-db/ip_db.py | Added `handle_stale_days` call and improved IP address parsing by adding `.strip()` calls to handle potential whitespace. |
| netflow-db/spectrum_db.py | Added `handle_stale_days` call and refactored IP parsing to avoid calling `.strip()` twice. |
| netflow-db/structure_db.py | Added `handle_stale_days` call and refactored IP parsing to avoid calling `.strip()` twice. |

</details>



<h3>Sequence Diagram</h3>

```mermaid
sequenceDiagram
    participant User
    participant ProcessDB as process_pending_files()
    participant HandleStale as handle_stale_days()
    participant GetStale as get_stale_days()
    participant ResetDay as reset_day_for_reprocessing()
    participant DB as SQLite Database
    
    User->>ProcessDB: Start processing
    ProcessDB->>HandleStale: Check for stale days
    HandleStale->>GetStale: Find mixed-status days
    GetStale->>DB: Query processed_files<br/>GROUP BY router, day
    DB-->>GetStale: Return stale days
    GetStale-->>HandleStale: Set of (router, day_start)
    
    loop For each stale day
        HandleStale->>ResetDay: Reset(router, day_start)
        ResetDay->>DB: DELETE FROM stats tables<br/>(5m, 30m, 1h, 1d aggregates)
        ResetDay->>DB: UPDATE processed_files<br/>SET status = NULL
        DB-->>ResetDay: Return counts
        ResetDay-->>HandleStale: Stats deleted & files reset
    end
    
    HandleStale-->>ProcessDB: Summary with total counts
    ProcessDB->>DB: Process files with clean state
    ProcessDB-->>User: Complete with accurate aggregates
```

<!-- greptile_other_comments_section -->

<!-- /greptile_comment -->