Built using Tauri, it processes your entire listening history, as generated by Spotify, stores it in a local database, and generates visualized user trends and stats, giving you an extended view into your music tastes. A more complete Spotify Wrapped.
yarn install
yarn tauri dev
The user uploads their extended history zip file they get from Spotify directly into the application. The app then processes each JSON file in the zip and generates its own set of processed data in a local SQLite database, saved on the user's filesystem. Then the user can navigate through the various pages showing yearly or monthly stats.
The application displays aggregated information such as total number of tracks/time listened over time, along with a list of Top Tracks/Artists/Albums for each year/month. The user can also see specific stats about a track or an artist, such as first/total listens, discovery date, etc
The frontend is written using Vue.js. It uses vue-router for application routing and navigation, and pinia for state management, caching any data of previously visited pages in the store.
The backend, including the data processeing layer is written using Rust, and utilizes the SQLite and Tauri store plugin to keep track of the processed data.
The first version of the data processing was written as one giant TypeScript file that went through each file in the raw listening history data, and precomputed all the yearly and monthly statistics, and stored in the user's filesystem as processed files for easy access. It was all done as one long-running script right in the UI part of the application, the idea being that listening history doesn't change, and the user only needs to do this once.
However, any interruption, whether it be an app crash or user navigating away, left the data in an odd state, with partially saved processed files. Reuploading would lead to double counting and corrupt files. This, along with my desire to learn more about Rust, resulted in a migration to Rust. Using Rust's spawn_blocking, and the Tauri store and temp file, I made the processing step asynchronous, safer and more robust, with status tracking and crash recovery. The app was able to resume the data processing safely if there was any interruption. If any processed files already existed, the code knew to read the relevant existing files and merge the new data into them, carefully avoiding double counting anything.
After going to bed happily on a late Sunday night, I woke up on Monday to realize how tunnel visioned I'd been in my data processing implementation. I was so focussed on making it work in Rust that I didn't think about if it was even the right solution. Or that a real solution is just a simple database. I used JSON files to make a poor man's database, using temp files as a replacement for database transactions. All I need was something simple like SQLite, to store a db file on the user's sytesm, to query, filter, sort or aggregate any data that I need. All the logic I wrote to carefully and systematically process the raw data was replaced by simply shoving it all in a database, and running queries on it. All I needed was a good schema, some useful indexes and uniqueness constraints, transactions, and the problem solved itself. It was painful to delete my hundreds of line of code but it was the right move.