Jetty Start Mechanism

Make sure you have read the Jetty architecture section if you are not familiar with the terms used in this section.

The Jetty start mechanism is invoked by executing $JETTY_HOME/start.jar, from within a $JETTY_BASE directory, with zero or more command line options:

$ cd $JETTY_BASE
$ java -jar $JETTY_HOME/start.jar ...
bash

The Jetty start mechanism has two main modes of operation:

  • The tool mode, detailed in this section, when it is used as a command line tool to configure the $JETTY_BASE directory by enabling modules, creating sub-directories and files, downloading files, etc. In this mode, the JVM started with java -jar $JETTY_HOME/start.jar performs the specified command and then exits.

  • The start mode, detailed in this section, when it is used to start the JVM that runs Jetty with the specified configuration. In this mode, the JVM started with java -jar $JETTY_HOME/start.jar starts Jetty and does not exit until stopped, for example by hitting Ctrl+C on the terminal.

Refer to the Jetty start mechanism reference section for the complete list of the available command line options.

You want to use the Jetty start mechanism to configure your $JETTY_BASE and then to start Jetty.

Configuring $JETTY_BASE

Within the Jetty start mechanism, the source of configurations is layered in this order, from higher priority to lower priority:

  • The command line options.

  • The $JETTY_BASE directory, and its files.

  • The directory specified with the --add-config-dir option, and its files.

  • The $JETTY_HOME directory, and its files.

Enabling Modules

You can enable Jetty modules persistently across restarts with the --add-modules command:

$ java -jar $JETTY_HOME/start.jar --add-modules=server,http
bash

The Jetty start mechanism will look for the specified modules following the order specified above. In the common case (without a --add-config-dir directory), it will look in $JETTY_BASE/modules/ first and then in $JETTY_HOME/modules/.

Since the server and http modules are standard Jetty modules, they are present in $JETTY_HOME/modules/ and loaded from there.

When you enable a Jetty module, the Jetty start mechanism:

  • Creates the correspondent $JETTY_BASE/start.d/*.ini module configuration file. The content of these *.ini files is copied from the [ini-template] section of the correspondent *.mod file.

  • Executes the directives specified in [files] section (if present) of the *.mod file. This may simply create a file or a directory, or download files from the Internet. This step is performed transitively for all module dependencies.

For example, enabling the server and http modules results in the $JETTY_BASE directory to have the following structure:

$JETTY_BASE
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── http.ini
    └── server.ini

The $JETTY_BASE/resources/jetty-logging.properties is created by the [files] directives of the logging-jetty module, which is a transitive dependency of the server module.

Disabling Modules

A module is enabled because the correspondent $JETTY_BASE/start.d/*.ini file contains a --module=<name> directive.

Commenting out the --module=<name> directive effectively disables the module.

Deleting the correspondent $JETTY_BASE/start.d/*.ini file also disables the module.

Editing *.ini Files

You can now edit the $JETTY_BASE/start.d/*.ini configuration files, typically by uncommenting properties to change their default value.

The $JETTY_BASE/start.d/*.ini configuration file may be missing, if the correspondent module is a transitive dependency. You can easily generate the configuration file by explicitly enabling the module, for example to generate the $JETTY_BASE/start.d/logging-jetty.ini configuration file you would issue the following command (the module order does not matter):

$ java -jar $JETTY_HOME/start.jar --add-modules=server,http,logging-jetty
bash

The $JETTY_BASE directory structure is now:

$JETTY_BASE
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── http.ini
    ├── logging-jetty.ini
    └── server.ini

You want to edit the $JETTY_BASE/start.d/*.ini configuration files so that the configuration is applied every time Jetty is started (or re-started).

For example, $JETTY_BASE/start.d/http.ini contains the following property, commented out:

http.ini
# jetty.http.port=8080

You can change the clear-text HTTP port Jetty listens to by uncommenting that property and changing its value:

http.ini
jetty.http.port=9876

When Jetty is started (or re-started) this configuration is applied and Jetty will listen for clear-text HTTP/1.1 on port 9876.

Enabling Modules on Command Line

You can also enable a module transiently, only for the current execution of the java -jar $JETTY_HOME/start.jar command.

If you have an empty $JETTY_BASE, the following command enables the server and http modules, but does not create any $JETTY_BASE/start.d/*.ini files.

$ java -jar $JETTY_HOME/start.jar --module=server,http
bash

Since there are no $JETTY_BASE/start.d/*.ini files, you can only customize the properties via the command line, for example:

$ java -jar $JETTY_HOME/start.jar --module=server,http jetty.http.port=9876
bash

Enabling modules on the command line is useful to verify that the modules work as expected, or to try different configurations.

It is possible to enable some module persistently via --add-modules and some other module transiently via --module.

Remember that once the current execution terminates, the modules enabled transiently on the command line via --module and their configuration are not saved and will not be enabled on the next execution (unless you specify them again on the command line).

Specifying Properties on Command Line

You can also specify module properties transiently, only for the current execution of the java -jar $JETTY_HOME/start.jar command.

Properties specified on the command line override the same properties specified in *.ini files.

For example, you can enable the server module in $JETTY_BASE and the http module on the command line, and specify properties for both on the command line:

# Enable the server module in $JETTY_BASE.
$ java -jar $JETTY_HOME/start.jar --add-modules=server

# Start Jetty enabling the http module on the command line,
# and specifying properties for both modules.
$ java -jar $JETTY_HOME/start.jar --module=http jetty.server.dumpAfterStart=true jetty.http.port=9876
bash

Adding Your Own Modules

Refer to the custom module section for the details about how to create your own modules.

You can add your own modules by adding a $JETTY_BASE/modules/*.mod file.

For example, you may want to add a Postgres JDBC driver to the server class-path, to avoid that each deployed web application bring its own version. This allows you to control the exact Postgres JDBC driver version for all web applications.

Create the $JETTY_BASE/modules/postgresql.mod file:

postgresql.mod
[description]
Postgres JDBC Driver Module

[lib]
lib/postgresql-${postgresql-version}.jar

[files]
maven://org.postgresql/postgresql/${postgresql-version}|lib/postgresql-${postgresql-version}.jar

[ini]
postgresql-version?=42.6.0

[ini-template]
## Postgres JDBC version.
# postgresql-version=42.6.0

Then enable it:

$ java -jar $JETTY_HOME/start.jar --add-modules=postgresql
bash

Enabling the postgresql module will execute the [files] directive (downloading the *.jar file from Maven Central if not already present) and create the $JETTY_BASE/start.d/postgresql.ini with the content of the [ini-template] section.

The [lib] section ensures that the specified file is in the server class-path when Jetty is started.

You can display the Jetty configuration to verify that the server class-path is correct.

Custom Module with JVM Options

Using a custom Jetty module, you can customize the JVM startup options.

This is useful if you need to start Jetty and want to specify JVM options such as:

  • -Xmx, to specify the max heap size

  • -Xlog:gc, to specify the GC log file and options

  • -javaagent, to specify Java agents

  • -XX: options, for example to specify the GC implementation

  • --enable-preview, to enable Java preview features

Start by creating $JETTY_BASE/modules/jvm.mod:

jvm.mod
[description]
JVM Options Module

[exec]
-Xmx1g
-Xlog:gc*,gc+stats=off:file=logs/gc.log:time,level,tags

Enable it:

$ java -jar $JETTY_HOME/start.jar --add-modules=jvm
bash

Since the module defines an [exec] section, it will fork another JVM when Jetty is started.

This means that when you start Jetty, there will be two JVMs running: one created by you when you run java -jar $JETTY_HOME/start.jar, and another forked by the Jetty start mechanism with the JVM options you specified (that cannot be applied to an already running JVM).

Again, you can display the JVM command line to verify that it is correct.

The second JVM forked by the Jetty start mechanism when one of the modules requires forking, for example a module that contains an [exec] section, may not be desirable, and may be avoided as explained in this section.

Displaying the Configuration

Once you have enabled and configured the $JETTY_BASE, you can display the configuration to verify that it is correct.

Using the standard server and http Jetty modules, and the postgresql and jvm custom Jetty module defined above, you obtain:

$ java -jar $JETTY_HOME/start.jar --list-config
bash
[jetty]
setupModules=code:example$jetty-modules/jvm.mod,code:example$jetty-modules/postgresql.mod
setupArgs=--add-modules=server,http,postgresql,jvm
args=--list-config

Note how the configuration displayed above includes:

  • In the list of enabled modules, the postgresql and jvm modules

  • In the list of JVM arguments, those specified by the jvm module

  • In the server class-path, the *.jar file specified by the postgresql module

Displaying the JVM Command Line

The Jetty start mechanism can display a full JVM command line that will start Jetty with the configuration you specified, with the --dry-run option:

$ java -jar $JETTY_HOME/start.jar --dry-run
bash

The full JVM command line generated by --dry-run can be split in various parts that can be used individually, for example in scripts.

Furthermore, Jetty modules may specify the --exec option that will fork a second JVM to start Jetty, which may not be desirable. Some option, such as --jpms, imply --exec, as it won’t be possible to modify the module-path in the already started JVM.

To start Jetty without forking a second JVM, the --dry-run option can be used to generate a command line that is then executed so that starting Jetty only spawns one JVM.

You can use the --dry-run option as explained below to avoid forking a second JVM when using modules that have the [exec] section, or the --exec option, or when using the --jpms option.

For example, using the --dry-run option with the jvm.mod introduced in this section produces the following command line:

$ java -jar $JETTY_HOME/start.jar --dry-run
bash
[jetty]
setupModules=code:example$jetty-modules/jvm.mod
setupArgs=--add-modules=http,jvm
args=--dry-run
replace=( ),$1\\\n

You can then run the generated command line.

For example, in the Linux bash shell you can run it by wrapping it into $(...):

$ $(java -jar $JETTY_HOME/start.jar --dry-run)
bash

The --dry-run option is quite flexible and below you can find a few examples of how to use it to avoid forking a second JVM, or generating scripts or creating an arguments file that can be passed to (a possibly alternative) java executable.

To display the java executable used to start Jetty:

$ java -jar $JETTY_HOME/start.jar --dry-run=java
bash
[jetty]
setupArgs=--add-modules=http
args=--dry-run=java

To display the JVM options:

$ java -jar $JETTY_HOME/start.jar --dry-run=opts
bash
[jetty]
setupModules=code:example$jetty-modules/jvm.mod
setupArgs=--add-modules=http,jvm
args=--dry-run=opts
replace=( ),$1\\\n

To display the JVM class-path:

$ java -jar $JETTY_HOME/start.jar --dry-run=path
bash
[jetty]
setupModules=code:example$jetty-modules/postgresql.mod
setupArgs=--add-modules=http,jvm
args=--dry-run=path
replace=( |:),$1\\\n

To display the JVM class-path and module-path, if you want to start Jetty using JPMS with the --jpms option:

$ java -jar $JETTY_HOME/start.jar --jpms --dry-run=path
bash
[jetty]
setupModules=code:example$jetty-modules/postgresql.mod
setupArgs=--add-modules=http,jvm
args=--jpms --dry-run=path
replace=( |:),$1\\\n

To display the JVM main class:

$ java -jar $JETTY_HOME/start.jar --dry-run=main
bash
[jetty]
setupArgs=--add-modules=http
args=--dry-run=main

To display the JVM main class when starting Jetty using JPMS:

$ java -jar $JETTY_HOME/start.jar --jpms --dry-run=main
bash
[jetty]
setupArgs=--add-modules=http
args=--jpms --dry-run=main

The main class is typically Jetty’s XmlConfiguration class that accepts, as program arguments, a list of properties and a list of Jetty XML files to process. The Jetty XML files compose together the Jetty components that are then configured with the values from the command line properties.

To display the program arguments passed to the main class:

$ java -jar $JETTY_HOME/start.jar --dry-run=args
bash
[jetty]
setupModules=code:example$jetty-modules/postgresql.mod
setupArgs=--add-modules=http
args=--dry-run=args
replace=( ),$1\\\n

Note how the program arguments are a list of properties in the form <name>=<value> and a list of Jetty XML files.

The various parts of the full JVM command line can be combined to leverage the arguments file feature (that is, specify the JVM options in a file rather than on the command line) that is built-in in the java executable:

$ java -jar $JETTY_HOME/start.jar --dry-run=opts,path,main,args > /tmp/jvm_cmd_line.txt
$ /some/other/java @/tmp/jvm_cmd_line.txt
bash

Using --dry-run=opts,path,main,args can be used to avoid that the Jetty start mechanism forks a second JVM when using modules that require forking:

$ java $(java -jar $JETTY_HOME/start.jar --dry-run=opts,path,main,args)
bash

The output of different --dry-run executions can be creatively combined in a shell script:

$ OPTS=$(java -jar start.jar --dry-run=opts,path)
$ MAIN=$(java -jar start.jar --dry-run=main)
$ ARGS=$(java -jar start.jar --dry-run=args)
$ java $OPTS -Dextra=opt $MAIN $ARGS extraProp=value extra.xml
bash

Starting Jetty

After you have configured the $JETTY_BASE directory, as explained in this section, you can start Jetty as a standalone server.

In the start mode, the Jetty start mechanism computes a JVM command line with JVM options, system properties, class-path, module-path, main class and program arguments, and then executes it, forking a new JVM if necessary.

The Jetty start mechanism performs these steps:

  1. Loads all the Jetty modules files (that have extension *.mod) from the modules/ subdirectory of each configuration source directory (see this section for the list of configuration sources). In this way, a Jetty module graph can be built in memory, where the module dependencies form the edges of the graph and each node contains the metadata information declared by each module (for example, the libraries that it needs, the XML files to process, and so on), in preparation for the next step.

  2. Reads the Jetty module configuration files (that have extension *.ini) from the start.d/ subdirectory of each configuration source directory and from the command line. This step produces a list of enabled modules; for each enabled module all its dependencies are transitively resolved by navigating the graph built in the previous steps.

  3. Processes the list of enabled (explicitly and transitively) modules, gathering the list of libraries to add to the class-path or module-path, the JPMS directives to add to the command line if necessary, the properties and XML files to add as program arguments, etc., so that a full JVM command line can be generated.

  4. Executes the command line, either in-JVM or by forking a second JVM (if the --exec option is present or implied by other options such as --jpms), and waits for the JVM, or the forked JVM, to exit.

Server Class-Path

When the Jetty server is started in-JVM, the server class-path gathered by processing the enabled modules is organized in a URLClassLoader, the Jetty Start ClassLoader, that is a child of the System ClassLoader:

Diagram

The System ClassLoader only has $JETTY_HOME/start.jar in its class-path, since the JVM was started with java -jar $JETTY_HOME/start.jar. The Jetty Start ClassLoader has in its class-path the *.jar files gathered by processing the enabled modules, typically from $JETTY_HOME/lib/jetty-*.jar, but possibly also from $JETTY_BASE/lib/*.jar if custom modules extend the server class-path with their own *.jar files.

When the Jetty server is started in a forked JVM, there will be two JVMs: one started by you with java -jar $JETTY_HOME/start.jar and one forked by the Jetty start mechanism. In the forked JVM, the System ClassLoader has the server class-path and/or module-path in its class-path, since the forked JVM is started with java --class-path $JETTY_HOME/lib/jetty-server-<version>.jar:...:

Diagram

It is worth mentioning that there are two standard Jetty modules that allow you to easily add entries to the Jetty server class-path:

  • The resources module, which adds the $JETTY_BASE/resources directory to the server class-path. This is useful if you have third party libraries that lookup resources from the class-path: just put those resources in the $JETTY_BASE/resources/ directory.
    Logging libraries often perform class-path lookup of their configuration files (for example, log4j.properties, log4j.xml, logging.properties, and logback.xml), so $JETTY_BASE/resources/ is the ideal place to add those files.

  • The ext module, that adds all the *.jar files under the $JETTY_BASE/lib/ext/ directory, and subdirectories recursively, to the server class-path.

    On one hand, the ext module provides a handy place to put third party libraries and their dependencies; on the other hand, the $JETTY_BASE/lib/ext/ directory may become a confused mixture of many *.jar files from different third party libraries.

    Prefer to group third party libraries and their dependencies into their own directories using custom modules, or at least group them into $JETTY_BASE/lib/ext/ subdirectories such as $JETTY_BASE/lib/ext/util/ or $JETTY_BASE/lib/ext/acme/.

Server Jakarta EE Environment Class-Path

When a Jakarta EE Jetty module is enabled, for example the ee10-webapp and/or the ee10-deploy module, the server ClassLoader hierarchy is modified as follows:

Diagram

When different versions of the Jakarta EE Jetty deploy or webapp modules are enabled, the server ClassLoader hierarchy is the following:

Diagram

Note how the Jetty EE 9 ClassLoader and the Jetty EE 10 ClassLoader are siblings. In this way, they can load independently their own version of the Jakarta EE APIs *.jar files, and the Jetty implementation of the Jakarta EE APIs, without conflicts.

Assembling Jetty Components

The Jetty start mechanism eventually invokes, by default, main class org.eclipse.jetty.xml.XmlConfiguration, passing properties and Jetty XML files as program arguments.

The Jetty XML files are nothing more than Java code in XML format.

The XML files are processed to instantiate Jetty components such as org.eclipse.jetty.server.Server or org.eclipse.jetty.util.ssl.SslContextFactory$Server. The components are then assembled together to provide the configured Jetty features.

The Jetty XML files are parametrized using properties, and a property is just a name/value pair.

This parametrization of the XML files allows an XML file that resides in $JETTY_HOME/etc/ to declare a property such as jetty.http.port, and allow this property to be set in a $JETTY_BASE/start.d/http.ini file, so that you don’t need to change the XML files in $JETTY_HOME, but only change *.ini files in your $JETTY_BASE.

You can write your own custom modules with your own Jetty XML files, and your own properties, to further customize Jetty.

Stopping Jetty

When Jetty is started, the Jetty components that you have configured by enabling Jetty modules are assembled and started.

If you have started Jetty from a terminal, you can exit the Jetty JVM by hitting Ctrl+C on the same terminal.

Similarly, from a different terminal, you can exit the Jetty JVM using kill -INT <pid> or kill -TERM <pid>.

In the three cases above, the JVM is exited, but by default Jetty components are not stopped. If you want to stop the Jetty components, to stop Jetty more gracefully, you can start Jetty with this property:

$ java -jar $JETTY_HOME/start.jar jetty.server.stopAtShutdown=true
bash

This property can also be set in $JETTY_BASE/start.d/server.ini so that it is persistently configured across Jetty restarts (see also the server module).

The jetty.server.stopAtShutdown property configures a JVM shutdown hook that is run, stopping the Server instance, when the JVM exits.

Obviously, the JVM can also be stopped with kill -KILL <pid> that exits the process abruptly without running the JVM shutdown hooks.

Stopping Gracefully

Stopping Jetty abruptly when there are active HTTP requests being handled may result in a variety or errors, because Jetty components that are used to handle requests are being stopped concurrently.

For example, when the Jetty thread pool is stopped, an attempt to submit a task would throw RejectedExecutionException; when a component is stopped, its fields may be nulled-out, resulting in a NullPointerException being thrown if the component is used; etc.

You can stop Jetty gracefully by adding the graceful Jetty module (see this section for more information).

When Jetty is stopped, the graceful module organizes to reject new requests, but allows existing requests to finish within a configurable timeout; then closes all the connections, stops all the Jetty components, and then exits the JVM. In this way, existing requests are not responded with an error caused by the server stopping, provided they complete within the timeout.

Stopping Jetty from Remote

You can configure a Jetty server so that it can be stopped by remote clients using a command sent through a TCP socket.

You can start Jetty with the following properties:

  • stop.host, the host name Jetty will bind to listen for stop commands. Defaults to 127.0.0.1 which means that the stop command can be issued only clients that run on the same host as Jetty.

  • stop.port, the port number Jetty will listen to for stop commands. Defaults to -1, which means that Jetty will not listen to any port.

  • stop.key, the password to verify when a stop command is received. Defaults to a password that is randomly generated and printed when Jetty starts.

$ java -jar $JETTY_HOME/start.jar stop.port=8181
bash
[jetty]
setupArgs=--add-modules=http
args=stop.port=8181
highlight=(?i)stop.key

In the example above, Jetty is started with just the stop.port property, and the stop.key is printed on the terminal when Jetty starts.

You can choose your own stop.key, but make sure it’s a strong password.

A remote client can now use the Jetty start mechanism to stop the remote Jetty server:

$ java -jar $JETTY_HOME/start.jar --stop stop.port=8181 stop.key=<stop.key>
bash

Note the --stop command along with the stop.port and stop.key properties. The stop.key must be the same as the one of remote Jetty server, either the one you chose, or the one printed on the terminal when Jetty starts.

Remote clients can wait for the remote Jetty server to shut down by specifying the stop.wait property with the number of seconds to wait:

$ java -jar $JETTY_HOME/start.jar --stop stop.port=8181 stop.key=<stop.key> stop.wait=15
bash

If the time specified elapses, without the confirmation that the remote Jetty server stopped, then the --stop command exits with a non-zero return code.

Start Mechanism Logging

The steps performed by the Jetty start mechanism are logged by the StartLog class, that outputs directly, by default, to System.err.

This is necessary to avoid that the Jetty start mechanism depend on logging libraries that may clash with those defined by Jetty logging modules, when Jetty is started in-VM.

This section is about the logging performed by the Jetty start mechanism before it configures and starts Jetty. See the logging section for information about logging when Jetty starts.

You can enable DEBUG level logging with the --debug command line option, for both the tool and start modes:

$ java -jar $JETTY_HOME/start.jar --debug ...
bash

You can send the start log output to a file, by default relative to $JETTY_BASE, with the --start-log-file=<file> option:

$ java -jar $JETTY_HOME/start.jar --debug --start-log-file=start.log ...
bash

This is useful for capturing startup issues where the Jetty-specific logger has not yet kicked in due to a possible startup configuration error.

Usage Reference

[jetty]
args=--help