Running tests

Run tests on Lute

After luau-run init, run the tests from the project’s folder:

luau-run test                          # every test under the configured roots
luau-run test tests\Arithmetic.spec.luau
luau-run test --workers 4              # up to 4 workers
luau-run test --isolated --workers 2   # a fresh process for each file

--progress (or progress under [test]) sets whether each file gets a line on standard error as it finishes: auto (the default), always, never, or failures, which shows only files that did not pass. auto and failures print only at a terminal.

Specs for Roblox Studio run the same way, through the Roblox Studio lane.

Workers and isolation

The default is one worker. worker-count under [test] sets another, and --workers overrides it for one run, from 1 to 64.

Jest specs are split round-robin into one group per worker, and each worker runs its group one file after another in one process. --isolated gives each file a fresh process instead. Script tests and custom runners always run each file in a process of its own. A failure is not retried.

The Studio lane runs one Studio at a time, so its specs share one session whatever --workers says. See Files in one session.

Time limits

• spec-timeout-seconds limits each test file, 120 by default.
• when-spec-times-out sets what happens after a file times out: skip (the default) abandons the file and goes on, stop ends its process, and restart starts a fresh process for the files after it.
• A file that never yields, such as one stuck in a busy loop, cannot be abandoned. Its process is ended 10 seconds after its limit, and unless when-spec-times-out is restart, the files after it in that process do not run. The report names them.
• On the Studio lane, Studio has 60 seconds after it opens to start the tests. boot-timeout-seconds under [test.roblox-studio] changes it.
• timeout-seconds limits the whole run. Unset, the per-file limits bound the run, so a healthy run is never cut short.
• Jest’s own per-test timeout still applies inside a spec: 5 seconds, unless the test passes a third argument, it(name, fn, milliseconds).

Stop a run

Press Ctrl+C or Ctrl+Break once to stop a run. Luau Run ends its test processes and any Studio it started, removes its temporary files, prints the report so far and exits 2. A Studio you opened yourself is left alone. Press either again to end Luau Run at once, which leaves those files behind.

The report lists each file that did not finish under “the run was cancelled”. A code under it, such as studio.session-failed, is what the stopped process reported as it was ended.

A test that sends Ctrl+C or Ctrl+Break to its own console ends only its own file. The other files keep running.

What a test leaves running

On Windows, everything a script test or a Jest runner on Lute starts ends with it. When the test process exits by itself, Luau Run ends anything still running a second later and lists it after the report:

tests/Server.test.luau: warning[test.left-process-running]: The test process for this file exited, and Luau Run ended what it left running: PING.EXE.

The file’s result stands, even if ending those programs runs past its time limit. The warning names up to 8 programs, then says how many more there were. When a file is stopped at its limit or by Ctrl+C, Luau Run ends everything it started at once and names nothing.

Read the results

luau-run test prints its report when every file has finished:

FAIL  tests/Arithmetic.spec.luau:6
  fails on purpose
  Assertion (toBe): expect(received).toBe(expected) -- Object.is equality

  Expected: 3
  Received: 2

FAILED  1 failed, 2 passed (3)

A failure names its file and line, on Lute and in Studio, then the test and the difference. The last line sums up the run:

Last line starts with	Meaning
PASS	Every test that ran passed.
FAILED	A test failed, a file failed outside its tests, or a file had no tests.
REFUSED	Luau Run refused some or all of the files. Each refusal gives its code and what to do.
INCOMPLETE	Some files did not finish. The lines above say which and why.
INVALID	The result cannot be trusted, such as a time limit hit after every file finished.

The counts are failed, passed and skipped, then, in parentheses, the number of tests selected. Skipped counts tests that .skip, .todo or a leftover .only kept from running, and a run with skips still passes: PASS 4 tests, 4 skipped.

• The first 10 failures are shown in full, and the rest are counted.
• A failed file shows the last 10 lines its test printed when it ran in a process of its own: script tests always, and Jest specs with --isolated.
• A spec whose requires do not fit its lane, such as a spec on Lute that requires Roblox instance paths, gets a warning before the run starts.
• In a failure’s text, a control character other than tab and newline, or a byte that is not UTF-8, prints as \xNN, such as \x1B.
• NO_COLOR turns colour off, and FORCE_COLOR turns it on outside a terminal.

When test refuses before anything runs, it prints one line, or one JSON object in a pipe or CI:

error[test.discovery.root-invalid]: Test root does not exist: tests/Wait10.spec.luau

Refusals gives each code and what to do. The report is written for people, so a CI job should read the exit code. A JUnit report is on the roadmap.

Exit codes

Exit	Meaning
0	Every selected test passed or was skipped.
1	A test failed, a file failed outside its tests, or a file had no tests.
2	The run was cancelled, or hit a time limit, its own or a file’s.
3	A test process could not start, ended without a result Luau Run could use, or was not cleaned up after. For example, Studio stopped before the files ran.
4	Luau Run refused a file, a lane or the whole run, such as for a missing tool or when no file was selected.
64	Usage or configuration error. No test ran.
70	Internal Luau Run error. No result was produced.

When a run has more than one outcome, the exit is the first of 3, 2, 4, 1 and 0 that applies. A run with failing tests and one refused file exits 4, so read the report too.

Every command can exit 64 or 70. Refusals gives the other commands’ exit codes.