I love being outdoor walking, running and biking. To track my activities, I have been using Strava since 2022 and before that I was using Nike Run Club. However, free subscription to these kind of apps do not provide creative visualizations. Instead of paying for subscription, I created a compact data pipeline in Python that processes GPX and FIT files (this is how tracking/fitness apps store your actitivies), and creates KML files with geo-location information and JSON files with aggregate statistics. These artifacts are then uploaded to AWS S3 and could be made public and fetched by other services.
In my case, I am using this data pipeline to power my personal website where I showed my walking, running and biking routes in the cities I have lived in: avcu.github.io/tracks.
- Activities from different resources (Strava, Nike Run Club, Garming) are normalized and saved as GPX files
- Aggregate statistics are calculated for each city that is provided by the user and saved as JSON files
- KML files with geolocation information are generated for each city
- Both KML files and JSON files are uploaded to AWS S3
- My personal website fetches the artifacts from AWS S3
- KML files are provided to Google Maps JavaScript API for tracks to be displayed on the maps based on the selected city
Additionally, I used Apache Airflow in this project which allows me to manually trigger the tasks on the UI, to create a recurring time scheduler or to trigger the tasks with new activities.
Airflow DAG UI:
