Set up ListenBrainz Server development environment¶
To contribute to the ListenBrainz project, you need a development environment. With your development environment, you can test your changes before submitting a patch to the project. This guide helps you set up a development environment and run ListenBrainz locally on your workstation. By the end of this guide, you will have…
- Installed system dependencies
- Registered a MusicBrainz application
- Initialized development databases
- Running ListenBrainz Server
ListenBrainz is hosted on GitHub at https://github.com/metabrainz/listenbrainz-server/.
You can use
git to clone it to your computer
git clone https://github.com/metabrainz/listenbrainz-server.git
ListenBrainz uses Docker for development. This helps you to easily create your development environment. Therefore, to work on the project, you first need to install Docker. If you haven’t already, follow the docker installation instructions for your platform.
Register a MusicBrainz application¶
Next, you need to register your application and get an OAuth token from MusicBrainz. This allows you to sign into your development environment with your MusicBrainz account.
- Name: (any name that you want and will recognize, e.g.
- Callback URL:
After entering this information, you’ll have an OAuth client ID and OAuth client secret. You’ll use these for configuring ListenBrainz.
With your new client ID and secret, update the ListenBrainz configuration file. If this is your first time configuring ListenBrainz, copy the sample to a live configuration.
cp listenbrainz/config.py.sample listenbrainz/config.py
Next, open the file with your favorite text editor and look for this section.
# MusicBrainz OAuth MUSICBRAINZ_CLIENT_ID = "CLIENT_ID" MUSICBRAINZ_CLIENT_SECRET = "CLIENT_SECRET"
Update the strings with your client ID and secret. After doing this, your ListenBrainz development environment is able to authenticate and log in from your MusicBrainz login.
To use the Last.fm importer you need an API account at Last.fm. You can
register for one at the Last.fm API page. Look for the following section in
# Lastfm API LASTFM_API_URL = "https://ws.audioscrobbler.com/2.0/" LASTFM_API_KEY = "USE_LASTFM_API_KEY"
LASTFM_API_KEY field with your Last.fm API key.
You also need to update the
API_URL field value to
To use the Spotify importer you need to register an application on the
Spotify Developer Dashboard. Use
as the callback URL.
After that, fill out the Spotify client ID and client secret in the following section of the file.
# SPOTIFY SPOTIFY_CLIENT_ID = '' SPOTIFY_CLIENT_SECRET = ''
The hostname on the callback URL must be the same as the host you use to
access your development server. If you use something other than
should update the
SPOTIFY_CALLBACK_URL field accordingly.
To use the CritiqueBrainz reviewer, you’ll need to visit the CritiqueBrainz applications page
and create/register an application. Use
http://localhost:8100/ as the homepage URL and
as the callback URL.
After registering, update the CritiqueBrainz section of the file with the client ID and client secret you obtained.
# CRITIQUEBRAINZ CRITIQUEBRAINZ_CLIENT_ID = '' CRITIQUEBRAINZ_CLIENT_SECRET = '' CRITIQUEBRAINZ_REDIRECT_URI = 'http://localhost:8100/profile/music-services/critiquebrainz/callback/'
Again, if you use something other than
localhost as the host you use to access your development server,
you should update the
Authorization callback URL fields accordingly when registering on CritiqueBrainz.
Initialize ListenBrainz containers¶
in the root of the repository. Using
docker-compose, this will build multiple
Docker images for the different services that make up the ListenBrainz server.
The first time you run this script it might take some time while it downloads all of the required dependencies and builds the services.
Initialize ListenBrainz databases¶
Your development environment needs some specific databases to work. Before proceeding, run these commands to initialize the databases.
./develop.sh manage init_db --create-db ./develop.sh manage init_msb_db --create-db ./develop.sh manage init_ts_db --create-db
Your development environment is now ready. Now, let’s actually see ListenBrainz load locally!
Run the magic script¶
Now that the databases are initialized, you can start your development
environment by running
By default, the web service listens on port 8100. If you already have a service listening
on this port, then you can change it by updating the ports section of
ports: - "8100:80"
To change the listening port, change only the value before the “:” to the port of your choice
and point your browser to
You will see the output of
docker-compose. You can shut down listenbrainz
by pressing CTRL^C. Once everything is running, visit your new site in a browser!
Look at the develop.sh documentation for more details.
A listenbrainz development environment contains a number of different containers running different services. We provide a small description of each container here:
db: A PostgreSQL server that contains data about users
redis: A redis server to store temporary server data
timescale: A PostgreSQL server with the TimescaleDB extension that stores users listens
rabbitmq: Used for passing listens between different services
web: This is the main ListenBrainz server
api_compat: A Last.fm-compatible API server
websockets: A websocket server used for the user-following and playlist updates on the front-end
If you add new python dependencies to ListenBrainz by adding them to
requirements.txt you will have
rebuild the web server. Use
./develop.sh build web
to do this.
./develop.sh build static_builder
Test your changes with unit tests¶
Unit tests are an important part of ListenBrainz. It helps make it easier for developers to test changes and help prevent easily avoidable mistakes later on. Before committing new code or making a pull request, run the unit tests on your code.
This builds and runs the containers needed for the tests. This script configures test-specific data volumes so that test data is isolated from your development data.
To run tests faster, you can use some options to start up the test infrastructure once so that subsequent running of the tests is faster:
./test.sh -u # build unit test containers, start up and initialise the database ./test.sh # run tests, do this as often as you need to ./test.sh -s # stop test containers, but don't remove them ./test.sh -d # stop and remove all test containers
If you made any changes to the frontend, you can run the tests for frontend using
You can also make use of the following frontend testing options for efficient testing.
./test.sh fe run frontend tests ./test.sh fe -u run frontend tests, update snapshots ./test.sh fe -b build frontend test containers ./test.sh fe -t run type-checker
Also, run the integration tests for ListenBrainz.
When the tests complete, you will see if your changes are valid or not. These tests are a helpful way to validate new changes without a lot of work.
Lint your code¶
ListenBrainz uses ESLint to lint the frontend codebase as part of the development process, in Webpack.
ESLint will automatically fix trivial issues and list all other issues in your terminal. Make sure to fix any error with the code you’ve modified.
There can be quite a lot of logs in the terminal, so if you want to look only at front-end build output, you can use this command to inspect only the static_builder logs:
./develop.sh logs -f static_builder
We provide a utility to wrap docker-compose and some common development processes.
To open a psql session to the listenbrainz database, run:
To open a psql session to the timescale database containing user listens, run:
To open a bash shell in the webserver container, run:
To open flask shell in the webserver container using ipython with the listenbrainz app loaded, run:
To open a redis shell:
develop.sh provides a direct interface to invoke manage.py inside a docker container.
manage.py is a click script containing a number of listenbrainz management commands.
To invoke manage.py, run:
./develop.sh manage <command>
To get a list of manage.py commands, run:
./develop.sh manage --help
To pass any other command to docker-compose, run:
To get a list of valid docker-compose commands, see the output of