How Does Traktomizer Audio Analyzer Work?

Traktomizer’s proprietary audio analysis algorithm provides incredibly detailed audio analysis across the full range of audio features measured by auto-playlist generating algorithms like Spotify, Apple Music, Google Music, Pandora and a hundred others.

Traktomizer also contains Genre-based truth-models of the audio-features to which music recommendation platforms are calibrated.

In tandem these features allow Traktomizer to benchmark your song’s audio vectors against the ranges deemed paramount (by the music industry) for making accurate recommendations and/or choosing the right songs for 3 billions unique playlists every day.

Traktomizer can calculate a song’s chart-worthiness and will even identify the most influential tweaks requiring the least change for the most gain; and every audio modification suggestion is made with respect to maintaining Artistic Freedom. Tweaks suggested by Traktomizer can only be implemented by the Producer(s), and not the program.

Ultimately Traktomizer focuses the Music Producer on audio properties showing up as weak relative to the song’s Genre. By strengthening those audio properties essential for accurate, automated, playlist-inclusion and song-suggesting, the ‘Traktomized’ music you publish is more likely favored and played. Compared with the increasingly inevitable probability that music you publish will forever be lost among 50+ million songs, Traktomizer helps you achieve the rather more desired outcome, every time.


Traktomizer in Operation – a Procedural Description

OK so ‘our song’ is nearly finished and we have a Premaster-mix on hand. Instead of sending this mix to the Mastering Studio as usual, we’ll let Traktomizer perform some audio analysis to identify opportunities for optimizations. So let’s begin…

Traktomizer Project Pallet

1. Traktomizer wants to compare our song with the 16 best it can find in our chosen Genre.
 Traktomizer allows us to include more songs, but adding too many will lower the overall standard and we only want to test against the cream, the most charted, the most played, the most loved music in recorded history – so let’s name these 16 our ‘Benchmark Analysis Pool’.

<= to the left you can see the Project Settings Pallet (click the image to enlarge it). Notice the modifiable but default settings. We just need to a) choose a song from disk to analyze; b) enter a Benchmark song Title (a song of the same genre we think is the best yardstick for a great mix); c) enter the Artist's Name; d) click 'Traktomize

2. Traktomizer Audio Analysis begins and a Progress Viewport opens. The viewport presents us with ongoing activity so we can see what’s happening.

3. Traktomizer sets about its first order of business: Analyzing our song. Then using our benchmark song as the seed Traktomizer will identify 16 benchmark songs. Traktomizer probes the Spotify API in search of benchmark songs and retrieves their audio properties Traktomizer is particularly interested in danceabilityacousticness, energy, liveness, loudness, speechiness, valence(mood), and duration. It takes Traktomizer only seconds to a few minutes (depending on your Internet connection speed) to probe, retrieve and compute all requisite audio vectors. Traktomizer will have also retrieved this week’s top 50 songs from Spotify and 7Digital in search of 5 currently charting songs in the same Genre as our benchmark song. And for each benchmark Artist, Traktomizer will identify their top ranking song. Within moments Traktomizer will have: a) retrieved the audio analysis for our song; b) retrieved the audio analysis for every benchmark song (from approx. 200 to 300 likely candidates) c) scrutinized this week’s 50 charting songs and possibly included and retrieved audio analysis for 5 more songs (currently charting and on-genre). Our final Analysis Pool should comprise our song (1) + the benchmark songs (17) + the charting songs (5)
 – these songs will be the very best, most recommended, most played, highest ranked songs in our Benchmark Artists’ Genre.

4. Traktomizer tallies, sorts and performs complex calculations, comparisons, aggregations and then appraises each of our song’s audio characteristics relative to the Benchmark Pool’s top songs and scores; then the top 3 best songs; the Pool’s mean/average and the Recommendation-bots ‘ideal-in-range’ properties. Traktomizer is pre-trained on song data from the Labrosa MIT 1 000 000 Song Data Set and the Billboard Year End Charts for the last 5 years, as well as EchoNest data. Traktomizer knows a lot about chart-ability. For example, at first glance it often seems more desirable to achieve scores which are higher than the Analysis Pool, but in reality it is often the lower score which is more desirable, i.e when the In-Range audio vector would become Out-of-Range as a result of being higher than the Benchmark’s value. Traktomizer is aware all of these relationships and makes calculations to ensure scores are properly accorded the correct weighting and status. Traktomizer assigns a point scoring system based on an index of audio vectors and social-media activity and their appropriate weight-based values. Some audio properties will score bonus points, which are usually awarded when several essential properties return positive measurements ‘as a group’. Within a few minutes Traktomizer will have: a) produced 20 beautifully animated SVG charts you can view while the analysis is in progress; b) our Dashboard View will have been created, complete with speedometer style gauges for visualizing our song’s strengths and weaknesses; c) an additional 7 Summary SVG charts to compliment the Dashboard View will also have been created. 5. We didn’t notice the spawning of an additional thread which started a Song Similarity Search based on our song’s Audio Vectors (rather than those of our Benchmark song ). This second thread will return around 50 more similar songs which Traktomizer scrutinizes for Tags, Keywords, Genres and Terms to identify which genres are most appropriate for our song.
6. Within a few moments Traktomizer will have produced a PDF Report listing the Song Title and Artist of each of the additional most similar matching songs based on our song’s audio properties, with a sorted list of the 15 most common terms and genres, in which, placement of our song would achieve greater discoverability and recommend-ability. On scrutiny of the Genres PDF, we will more than likely discover that our song is showing potential new popularity in various genres we never considered, the subsequent mentioning of which in blogs, news and the wide variety of available social media will generate additional plays and new fans, not to mention additional favor by the recommendation-bots. 7. Hopefully you’re sensing Traktomizer’s mojo, so while this has all been in-play, a thorough analysis of our entire song library was performed in another thread to determine our overall strongest to weakest music production skills across every song we’ve ever analyzed, including this new one. 8. Traktomizer also traversed all our audio analysis data, which is stored in its local database, and calculated our top 5 songs from all our songs i.e. our songs that display the most potential for charting and recommendations. And another sorted list has been produced naming our top 3 songs with the best audio characteristics, in each of 10 particular analysis vectors worth scrutinizing i.e. we now have an up-to-date list of the 3 songs with the highest Dance-ability, highest Energy, highest Valence, highest Acoustics etc. For each of these results a new SVG chart has been prepared. Traktomizer has also calculated our strongest and weakest production skills which can be seen at the Dashboard view. 9. In addition to the physical audio analysis data supplied by the EchoNest, Traktomizer also physically analyzed our song locally (even encoding our file on the fly if it needed to) and created and saved 8 detailed PNG decompositions to disk. We can view these using Traktomizer’s purpose-built, interactive Decompositions Gallery View (mouse-wheel enabled zooming and mouse-panning by dragging is also supported).

10. Traktomizer also generated 8 Open-GL charts and byte-array data-sets from its analysis. We can study these immersive, interactive views of our song, navigating our sound as one would play a 3D computer game. We can also export the OpenGL Charts to PNGs, at various scales.

11. Traktomizer produced 2 more PDF’s and saved them to our Documents folder. One of them is a copy of the Progress View Port trace which was being updated throughout this analysis task. The second PDF is a final summary report, which describes in detail how our song stacked up against all 22 benchmark songs. Much of the data contained in this report can be visualized on our Dashboard. We can use the Dashboard for a high-level overview, presented by various dials and summary charts. For a more granular and detailed look at how well our song performed, we would take a look at the Summary Report PDF which contains the individual values and totals for each audio vector. Additionally, the last portion of the report contains the updated song analysis results for our entire library, with our new song taken into account.

12. Traktomizer has determined our new song’s overall Chartability-Rating (see the Dashboard) and it’s weakest 3 audio properties (listed on the right side of the Dashboard.) It’s telling us that a) we should check these deficient areas and make appropriate changes if we think we can improve them, and b) making changes to these areas first is likely to achieve the maximum overall improvement possible (even if other areas were also found wanting attention).

13. Traktomizer sneaked in another process in search of all our benchmark song mp3 previews. It probably found previews for all the songs (but its not a given). A playlist was created for Traktomizer’s mp3 player. The playlist will be in the same order as our benchmark search. Clicking the topmost song in the playlist starts successive playback which will continue until we click the stop button.

14. Dynamic Range, Peak Level and RMS are also calculated: On the left and right side of our Dashboard, and recorded in Summary PDF Reports, are highly accurate readings detailing our songs dynamics and loudness characteristics. These were calculated locally by Traktomizer’s built-in routines. See the User Manual for a detailed description about how they should be interpreted.

15. Taking a quick visit to the Open-GL view we click the STFT Magphase Component Button-Icon on the View Title-bar. Traktomizer presents a render to show whether our song has demonstrated a continuous increase in loudness from beginning to end. Recent research seems to show that charting songs display a tendency towards increasing loudness. 16. Traktomizer encoded a number of byte-array files on disk. They are prefixed with our new song’s name and used by Traktomizer to reproduce the Open-GL charts at lightning speed, rather than reanalysing the audio file every time we want to visualize it. Additionally, the byte-arrays contain beat-matching, pitch, melody and a variety of array-like decompositions which can be used for creating remixes and new compositions in other programs. Planned future releases will take more advantage of these files. For now we generate them for use with the Open-GL viewer. If we make changes to our song we can perform an Iteration Analysis to plot our previous audio properties alongside our new version. We can easily see differences between each iterations, and Traktomizer will automatically refresh our OpenGL byte-array files.  Our Song  is ‘Traktomized’!

Traktomizer has many more features such as Loudness Benchmarking, Social Media Tracking & Metrics Benchmarking, Iterations Analysis, Custom Styles and GUI Layouts and Song Database Management.

~ A Work-in-Progress Example ~

Traktomizer has determined my song ‘Love Thing’ will benefit from tweaks to:

*Energy is weak: Pool = 0.705 | Love Thing = 0.593

*Duration too long: Pool = 237.83 s | Love Thing = 300.53 s

*Loudness very low: Pool = -5.8 dB | Love Thing = -12.7 dB

I checked the individual Audio Property Charts and the Traktomizer Dashboard. (see below)

Love Thing Mix Energy
Love Thing Mix Loudness
Love Thing Mix Duration

The Dashboard (below-left) confirmed tweaks to *Energy*, *Loudness* and *Duration* – displayed below the Chartability Rating, which was 21%.

I added a little Compression and EQ (between 500hz to 2.4khz) and after Iteration Analysis the updated Dashboard (below-right) reflects an improvement:

My Chartability Rating Doubled from 21% to 42%

The example revealed the dramatic impact of Energy, Loudness and Duration other Audio Properties remained largely unchanged


From the side-by-side Iteration Bar Chart below you can easily see a significant change in Energy resulting from my simple edit.

Love Thing Iteration 1

The remaining charts identify more focus for the next iteration of editing.

Love Thing Compared to my Benchmark Song

Love Thing Compared the Entire Song Analysis Pool of 21 Songs

Valence (emotion) | Speechiness | Liveness | Acousticness | instrumentalness | Danceability

lovething_benchmark lovething_compared_to_averages lovething_valence lovething_speechiness lovething_liveness lovething_acousticness lovething_instrumentalness lovething_danceability

Traktomizer also produced these interactive Open-GL decompositions

opengl_chroma_beatsync  opengl_harmonic_percussive opengl_hpss_percussive opengl_log_spec opengl_magphase  opengl_peakrms opengl_percussive_components opengl_spectrogram_log

Audio Decompositions were created the traditional way too.

Love Thing_mix_beat_sync  Love Thing_mix_chromagram Love Thing_mix_harmonic_percussive    Love Thing_mix_mfcc Love Thing_mix_logamplitude Love Thing_mix_rms Love Thing_mix_spectra_decompose  Love Thing_mix_melspectrogram