Markov Composer - Using machine learning and a Markov chain to compose music

Edit: If you want to see MarkovComposer in action, but you don't want to mess with Java code, you can access a web version of it here.


In the following article, I'll present some of the research I've been working on lately. Algorithms, or algorithmic composition, have been used to compose music for centuries. For example, Western punctus contra punctum can be sometimes reduced to algorithmic determinacy. Then, why not use fast-learning computers capable of billions of calculations per second to do what they do best, to follow algorithms? In this article, I'm going to do just that, using machine learning and a second order Markov chain.

A graph representing a Markov chain transition matrix (explained after) with 8 compositions learned.

Markov chain?

Markov chain, named after Andrey Andreyevich Markov (look, we even share the first name), is a (pseudo)random process of transition from one state to another. The transition is "memoryless" and it only depends on the current state and on the probabilities (saved in a so-called transition matrix). Sequence of events that preceeded the current state should in no way determine the transition. This "memorylessness" is also called Markov property. In short, transiting from one state to another is a random process based on probability.

The general idea

Markov chain is just plain perfect for algorithmic music compositions. Notes (128 of them) are used as possible states. For the implementation, I'm using a second order Markov chain, meaning two previous states (two previous notes) determine the next state and nothing else. All of the transition probabilites are stored in a 2^14x2^7 matrix. As input, the composer takes two integer values (0 <= n, m <= 127) representing 2 starting notes. Based on that the algorithm calculates/generates the next note and the generation process goes ad infinitum (until you stop it, that is). For the simplicity, currently, all the notes are played at the same pitch (127) and at the same time spacing between notes (300ms).

Calculating probabilities and weights for Markov chain transition matrix

Generating a Markov chain transition matrix (MCTM in future use) is based on a very simple principle. First of, a weight matrix is used (WM in future use). As I've stated before, since the algorithm is based on a second order Markov chain, the calculation process involves three notes. For every three notes, the first two are the "first" state and the third one is a second state, and therefore the following field WM[first_note*127+second_note][third_note] is just incremented by one. Of course, this is just the beginning of the process. After all of the weights have been incremented accordingly, and the WM is completely generated, it is "normalized"/converted to a MCTM by converting integer values to their percentage relative to the sum of all values in the row, effectively generating a complete MCTM. The code snippets below show both processes respectively. Both matrices are represented by a single one, called scoreMatrix.

WM generation

		public static void updateWeight(int n1, int n2, int n3) {

WM normalization/MCTM generation

		public static int sumAll(int pos) {
			int sum = 0;

			for(int i = 0; i < 128; sum+=scoreMatrix[pos][i++]);

			return sum;

		public static void normalizeMatrix() {
			for(int i = 0; i < 128*128; i++) {
				int sum = sumAll(i);
				if(sum != 0)
					for(int j = 0; j < 128; j++) 
						scoreMatrix[i][j] /= sum;					

The actual learning process

Learning process at the ends returns a generated WM matrix. After all of the learning is completed, the matrix is converted to a MCTM. Algorithm described in this article uses MIDI files (.mid) for learning. It processes the file note by note, updating the WM matrix accordingly, as described in the paragraph above. The iteration through the MIDI file is achieved by using Java's builtin Sequencer.

		public Learn(String midiName) {
			try {
				Sequence sequence = MidiSystem.getSequence(new File(midiName));

				int id[] = {0, 0, 0};
				int nArr[][] = new int[2][2];

				for(Track track : sequence.getTracks()) {
					for(int i = 0; i < track.size(); i++) { 				
						MidiEvent event = track.get(i);
						MidiMessage message = event.getMessage();
						if(message instanceof ShortMessage) {
							ShortMessage sm = (ShortMessage) message;

							if(sm.getCommand() == NOTE_ON) {
								int key = sm.getData1();

								for(int j = 0; j < 2; j++) {
									if(id[j] == 2) {
										id[j] = 0;
										Score.updateWeight(nArr[j][0], nArr[j][1], key);
									} else {
										nArr[j][id[j]++] = key;

			} catch(InvalidMidiDataException|IOException e) {

Choosing the right note

Yaay, we are on the most important part yet! All of the code above would be useless if we didn't have a way to generate the right note. The process is actually quite simple, and it's based on randomness, the current state (last two notes in the sequence), and, of course, the MCTM generated before. The chance is randomly generated using Java's builtin Math.random() function. Then, the aglorithm is iterating through the MCTM to find the note with the correct (closest) probability and returns it. (returns a value between 0 and 127)

		public static int nextNote(int n1, int n2) {
			double rnd = Math.random();
			double sum = 0.0;

			for(int i = 0; i < 128; i++) {
				sum += scoreMatrix[n1*127+n2][i];

				if(sum >= rnd)
					return i;

			return (int) (rnd*127);	/* In an off chance that no states are found (all have 0.0 probability of transition), the algorithm continues randomly */

Playing the output

The output is played by the Java's builtin Synthesizer, and the notes are generated on the go by the algorithm above, based on the previously generated MCTM. The output is mainly determined by the two starting notes, chosen either randomly or by the user.

		try {				
			Synthesizer synth = MidiSystem.getSynthesizer();;

			final MidiChannel[] channels = synth.getChannels();

			int fn, sn, nn;

			fn = n1;
			sn = n2;

			while (!this.isInterrupted()) {
				nn = Score.nextNote(fn, sn);

				int octave = (nn/12)-1;
				String noteName = NOTE_NAMES[nn%12];

				channels[0].noteOn(nn, Info.NOTE_VELOCITY);

				fn = sn;
				sn = nn;
		} catch(Exception e) {}

The final (alpha) product

An image representing the main screen of the application


Just an example what can Markov composer do. Based on a little bit of testing, there are probably a lot more, a lot better generated compositions.


Even though this is just an experiment, the results produced are not in total disharmony, and some starting combinations can produce pleasent sounding compositions.

The full code including the GUI is open-source and is publicly available at my github repository.