Gadget userguide
1 Introduction to Gadget
Gadget is the Globally applicable Area Disaggregated General Ecosystem Toolbox. Gadget is a flexible and powerful software tool that has been developed to model marine ecosystems, including both the impact of the interactions between species and the impact of fisheries harvesting the species. Gadget simulates these processes in a biologically realistic manner, and uses a framework to test the development of the modelled ecosystem in a statistically rigorous manner. Gadget has successfully been used to investigate the population dynamics of stock complexes in Icelandic waters, the Barents Sea, the North Sea and the Irish and Celtic Seas.
1.1 What is Gadget?
Gadget can run complicated statistical models which take many features of the ecosystem into account. Gadget works by running an internal forward projection model based on many parameters describing the ecosystem, and then comparing the output from this model to observed measurements to get a likelihood score. The model ecosystem parameters can then be adjusted, and the model re-run, until an optimum is found, which corresponds to the model with the lowest likelihood score. This iterative, computationally intensive process is handled within Gadget, using a robust minimisation algorithm. The Gadget software framework consists of three parts:
a parametric model to simulate the ecosystem
statistical functions to compare the model output to data
search algorithms to optimise the model parameters
Gadget allows the user to include a number of features of the ecosystem into the model: One or more species, each of which may be split into multiple components; multiple areas with migration between areas; predation between and within species; growth; maturation; reproduction and recruitment; multiple commercial and survey fleets taking catches from the populations.
Like any large piece of code, Gadget is based on previous ideas by several authors. Conceptually the modelling framework extends earlier multi species programming work such as MSVPA and MULTSPEC into a more generic statistical framework. Alternatively, Gadget is a conceptual extension of the Stock Synthesis statistical assessment single-species framework into a multi species setting.
The initial BORMICON code was developed as part of a multi species programme implemented at the Marine Research Institute in Reykjavik, Iceland, starting in 1992 with Ólafur K Pálsson as project coordinator and Gunnar Stefánsson coordinating the modelling work. Subsequently the code became the basis for Fleksibest at the Institute of Marine Research in Bergen, Norway. Further development work in 1999-2003 was partly funded by EU grant QLK5-CT1999-01609 (“Development of Structurally Detailed Statistically Testable Models of Marine Populations”). Development work for 2004-2006 was partly funded by EU grant 502482 (“Critical Interactions Between Species and their Implications for a Precautionary Fisheries Management in a Variable Environment - a Modelling Approach”). Developement work for 2010-2012 was partly funded by EU project “Defineit” under the MariFish ERA-NET (grant ERAC-CT-2006-025989)
The Gadget code is derived from the original BORMICON code of Halldór Narfi Stefánsson, Höskuldur Björnsson and Hersir Sigurgeirsson. Subsequent contributions include program additions by Morten Nygaard Åsnes and Kristin Frøysa to include the Fleksibest model. The ability to run in parallel across a network of computers using PVM was implemented primarily by Auðbjörg Jakobsdóttir with input from several other people, including Jón Guðmundsson who worked on a variant based on Condor. Several mathematical additions and changes to allow for parallel minimisation on a network were implemented by Þórdís Linda Þórarinsdóttir and Kristjana Ýr Jónsdóttir.
More recent changes include the work to use information from tagging experiments, which was implemented primarily by Sigurður Hannesson. The work on multivariate distributions was implemented by Bjarki Þór Elvarsson. Recent additions including major code cleanups, modifications to the input formats and debugging operations have been led by James Begley, who is the current maintainer of the code and coordinator of all programming efforts.
1.2 Getting Gadget
Gadget is distributed via GitHub and can be obtained at:
That page will have instructions for how to install the most recent version of Gadget. Follow the instructions there to install
Gadget is a program that runs on a Unix computing platform, and is regularly tested on machines running versions of Linux, Mac OSX and Windows. It should also run on other versions of Unix, but this may require modifications to the makefile in order to do so. Gadget is distributed as a set of source code files, which need to be compiled into an executable program before it can be run. Follow the instructions on the link above to fetch and compile the Gadget sources.
1.3 Running Gadget
Gadget is a command line program that runs on a Linux computing platform. To start Gadget, simply type the following at a standard Linux prompt:
gadget <options>where \(<\)options\(>\) are a combination of the starting switches described in the next section. Note that this assumes that the Gadget executable has been placed in a directory that is included in the $PATH, which might not be the case for some Linux distributions. If this is the case, Gadget should be started by including the path to the Gadget executable, for example by typing if the Gadget executable is in the current directory.
Gadget is not an interactive program, so once it has started there is no need for any further input from the user. However, if Gadget was started incorrectly (for example, using the wrong input files) then Gadget can be stopped by pressing \(<\)CTRL\(><\)C\(>\), which will interrupt the calculations, displaying a menu from which it is possible to either store the current calculations to a text file, or to exit (by pressing \(<\)Q\(>\)).
When Gadget starts, it will look for a file called “main”, which contains a list of all the other data files required. Gadget will search for this file in the following 2 locations:
the directory specified by the Linux environment variable called “GADGET_WORKING_DIR”, optionally taking data from the directory specified by the Linux environment variable called “GADGET_DATA_DIR”
the current directory (ie. from where Gadget has been started) if these environment variables have not been set (note that this is the default location)
When Gadget is reading in the input data files, these files will be checked to ensure that they are in the correct format, and if there is an error in the format Gadget will print an error message and stop. Note that when Gadget is checking the format of a data file containing a number of columns (eg. the “area” file) it will only check the first data line in the file (so, for the case of the “area” file it will check that there are 4 entries on the first row and then assume that all the other rows also have 4 entries). Hopefully, there should be enough information in these error messages to lead the user to the input file that is causing the error.
If all else fails, and the error messages do not lead the user to the source of the error, then there is an message board set up for any difficulties or questions that may arise, or reporting any bugs that get found. In addition to sorting out any problems that the user may be getting, we can also try to help with interpreting the output. This message board is located on the github repository for Gadget.
Sending an email to this address should get a response within a couple of working days. This message board can also be used to give us any feedback on the usefulness (or otherwise) of Gadget, so let us know what you think!
1.4 Starting Switches
The following is a list of the command line switches that Gadget will recognise, together with a brief description of what the switch means to the running of the Gadget model.
gadget -sStarting Gadget with the -s switch will start a simulation run, where a single run of the model will take place. This option is useful since it will give the model output (see Print Files, chapter 9), and it is used when running large Gadget models, using the paramin parallel processing optimiser to find the optimum.
gadget -lStarting Gadget with the -l switch will start a optimising run, where the overall likelihood score will be reduced to an optimum, depending on the optimising information given (see the -opt switch).
gadget -nStarting Gadget with the -n switch will start a network run, used in conjunction with the paramin optimiser to find an optimal solution for large models.
gadget -v
gadget --versionStarting Gadget with the -v (or --version) switch will display the version of Gadget that is being run.
gadget -h
gadget --helpStarting Gadget with the -h (or --help) switch will display a help screen, giving information about the various starting switches that can be used.
gadget -i <filename>Starting Gadget with the -i switch will give Gadget an inputfile file from which the initial values and bounds of any variables can be read (see Parameter Files, chapter 10, for more information on the format of this file).
gadget -opt <filename>Starting Gadget with the -opt switch will give Gadget an optimisation input file, from which the information about the optimisation routines will be read. This will specify the type of optimisation to perform, and also parameters for that optimisation routine (see Optimisation Files, chapter 11, for more information on the format of this file).
gadget -main <filename>Starting Gadget with the -main switch will specify a filename for the main Gadget model input file, which contains links to all the other data files that are to be used by Gadget (see Main File, section 3.1, for more information on the format of this file). If Gadget is started without the -main switch, Gadget will use the default filename “main” as the name for the main file.
gadget -p <filename>Starting Gadget with the -p switch specifies the file that Gadget will use to print the final values and bounds of the variables, in the same format as for the inputfile. This file can then be used as the starting point for a subsequent Gadget run, if required. This file will always be generated, and if Gadget is started without the -p switch the default filename is “params.out” which will be created in the current directory (see Output Files, chapter 12, for more information on the format of this file).
gadget -o <filename>Starting Gadget with the -o switch specifies the file that Gadget will use to print the score from the likelihood calculations. This file will give details on the parameters that have been used, the likelihood components that have been used, and the values for these parameters and likelihood components. This can be a large file if Gadget is performing an optimising run (see Output Files, chapter 12, for more information on the format of this file).
gadget -print <number>Starting Gadget with the -print switch will specify the frequency with which information is written to the likelihood output file (specified with the -o switch). The default value for this is 1, meaning that the likelihood information is written for every iteration.
gadget -precision <number>Starting Gadget with the -precision switch will specify the number of digits to be used when printing the output from the likelihood calculations to files specified with the -o switch.
gadget -log <filename>Starting Gadget with the -log switch will specify a file to which Gadget will write logging messages to keep a record of internal Gadget actions. This can be a large file if Gadget is performing an optimising run (see Log Output, section 12.3, for more information on the format of this file).
gadget -loglevel <number>Starting Gadget with the -loglevel switch will specify the level of detail that is to be used by Gadget when logging messages (both to the screen and to a log file if one has been specified with the -log switch). If Gadget is started without the -loglevel switch then the default is for warning messages and error messages to be shown during a simulation run (when Gadget is started with the -s switch) and only error messages are shown during an optimising run (when Gadget is started with the -l switch).
gadget -seed <number>Starting Gadget with the -seed switch will specify the value that is used to initialise the random number generator used within Gadget (see Repeatability, section 11.5 for more information on the use of the random number generator within Gadget runs).
gadget -m <filename>Starting Gadget with the -m switch will specify a file from which Gadget can read the other command line options. This file should contain a simple list of the switches and their values, as they would be entered from the command line.
gadget -printinitial <filename>Starting Gadget with the -printinitial switch will specify a file to which Gadget will write all internal information for the model at the start of the run (ie. the stock populations, likelihood calculations and other information from before the first timestep). This file will be large for moderately complicated models, and it is of most use for debugging purposes.
gadget -printfinal <filename>Starting Gadget with the -printfinal switch will specify a file to which Gadget will write all internal information for the model at the end of the run (ie. the stock populations, likelihood calculations and other information from after the last timestep). This file will be large for moderately complicated models, and it is of most use for debugging purposes.
gadget -maxratio <ratio>Starting Gadget with the -maxratio switch will specify the maximum ratio of prey that is allowed to be “consumed” on any one timestep. This consumption includes both the consumption by other stocks and the catch by any fleets. The default value is 0.95, which ensures that no more than 95% of the available stock biomass is consumed on a single timestep.
Most of these switches can be combined to specify more information about the Gadget run that will be performed. For instance:
gadget -l -i inputfile.txt -o likelihood.txt -opt optinfo.txt -print 10will start Gadget for a likelihood run, taking the initial values and information about the parameters from the file inputfile.txt, with the optimisation being done in accordance with the information in optinfo.txt, and printing likelihood information (every 10 iterations) to the file likelihood.txt.
Note that it is not possible to do both a simulation run and a likelihood run at the same time, and starting Gadget with both the -l and -s switches will result in a warning, after which Gadget will perform a simulation run, with the -l switch being ignored.