luna - curl/docs/tests/FILEFORMAT.md

  1<!--
  2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
  3
  4SPDX-License-Identifier: curl
  5-->
  6
  7# curl test suite file format
  8
  9The curl test suite's file format is simple and extendable, closely resembling
 10XML. All data for a single test case resides in a single ASCII file. Labels
 11mark the beginning and the end of all sections, and each label must be written
 12in its own line. Comments are either XML-style (enclosed with `<!--` and
 13`-->`) or shell script style (beginning with `#`) and must appear on their own
 14lines and not alongside actual test data. Test data files are syntactically
 15valid XML; lack of support for character entities is a big difference but macros
 16like %CR fill that particular role here.
 17
 18Each test case source exists as a file matching the format
 19`tests/data/testNUM`, where `NUM` is the unique test number, and must begin
 20with a `testcase` tag, which encompasses the remainder of the file.
 21
 22# Preprocessing
 23
 24When a test is to be executed, the source file is first preprocessed and
 25variables are substituted by their respective contents and the output version
 26of the test file is stored as `%LOGDIR/testNUM`. That version is what is read
 27and used by the test servers.
 28
 29## Base64 Encoding
 30
 31In the preprocess stage, a special instruction can be used to have runtests.pl
 32base64 encode a certain section and insert in the generated output file. This
 33is in particular good for test cases where the test tool is expected to pass
 34in base64 encoded content that might use dynamic information that is unique
 35for this particular test invocation, like the server port number.
 36
 37To insert a base64 encoded string into the output, use this syntax:
 38
 39    %b64[ data to encode ]b64%
 40
 41The data to encode can then use any of the existing variables mentioned below,
 42or even percent-encoded individual bytes. As an example, insert the HTTP
 43server's port number (in ASCII) followed by a space and the hexadecimal byte
 449a:
 45
 46    %b64[%HTTPPORT %9a]b64%
 47
 48## Hexadecimal decoding
 49
 50In the preprocess stage, a special instruction can be used to have runtests.pl
 51generate a sequence of binary bytes.
 52
 53To insert a sequence of bytes from a hex encoded string, use this syntax:
 54
 55    %hex[ %XX-encoded data to decode ]hex%
 56
 57Other bytes within the brackets that not percent encoded are inserted as-is.
 58
 59For example, to insert the binary octets 0, 1 and 255 into the test file:
 60
 61    %hex[%00%01%FF]hex%
 62
 63## Repeat content
 64
 65In the preprocess stage, a special instruction can be used to have runtests.pl
 66generate a repetitive sequence of bytes.
 67
 68To insert a sequence of repeat bytes, use this syntax to make the `<string>`
 69get repeated `<number>` of times. The number has to be 1 or larger and the
 70string may contain `%HH` hexadecimal codes:
 71
 72    %repeat[<number> x <string>]%
 73
 74For example, to insert the word hello 100 times:
 75
 76    %repeat[100 x hello]%
 77
 78## Whitespace
 79
 80To force CRLF newline, add this macro to the end of the line:
 81
 82    %CR  - carriage return
 83
 84To add significant whitespace characters at the end of the line, or to empty
 85lines:
 86
 87    %SP  - space
 88    %TAB - horizontal tab
 89
 90## Special characters
 91
 92Macros to help keep data files XML-compliant:
 93
 94    %AMP - Ampersand: `&`
 95    %GT  - Greater-than sign: `>`
 96    %LT  - Less-than sign: `<`
 97
 98## Insert capped epoch days
 99
100Mostly to test capped cookie expire dates: `%days[NUM]` inserts the number of
101seconds for the given number of days into the future, aligned to the nearest
102minute. That is the same calculation the cookie engine uses to cap expiration
103dates.
104
105## Include file
106
107This instruction allows a test case to include another file. It is helpful to
108remember that the ordinary variables are expanded before the include happens
109so `%LOGDIR` and the others can be used in the include line.
110
111The filename cannot contain `%` as that letter is used to end the name for
112the include instruction:
113
114    %include filename%
115
116Or, a variant of the above where the file is loaded as a newline-agnostic
117text file, and whitespace, special character macros and variables expanded
118after inclusion:
119
120    %includetext filename%
121
122## Conditional lines
123
124Lines in the test file can be made to appear conditionally on a specific
125feature (see the "features" section below) being set or not set. If the
126specific feature is present, the following lines are output, otherwise it
127outputs nothing, until a following else or `endif` clause. Like this:
128
129    %if brotli
130    Accept-Encoding
131    %endif
132
133It can also check for the inverse condition, so if the feature is *not* set by
134the use of an exclamation mark:
135
136    %if !brotli
137    Accept-Encoding: not-brotli
138    %endif
139
140You can also make an "else" clause to get output for the opposite condition,
141like:
142
143    %if brotli
144    Accept-Encoding: brotli
145    %else
146    Accept-Encoding: nothing
147    %endif
148
149Nested conditions are supported.
150
151# Variables
152
153When the test is preprocessed, a range of "variables" in the test file is
154replaced by their content at that time.
155
156Available substitute variables include:
157
158- `%CLIENT6IP` - IPv6 address of the client running curl (including brackets)
159- `%CLIENT6IP-NB` - IPv6 address of the client running curl (no brackets)
160- `%CLIENTIP` - IPv4 address of the client running curl
161- `%CURL` - Path to the curl executable
162- `%DATE` - current YYYY-MM-DD date
163- `%DEV_NULL` - Null device (e.g. /dev/null)
164- `%FILE_PWD` - Current directory, on Windows prefixed with a slash
165- `%FTP6PORT` - IPv6 port number of the FTP server
166- `%FTPPORT` - Port number of the FTP server
167- `%FTPSPORT` - Port number of the FTPS server
168- `%FTPTIME2` - Timeout in seconds that should be sufficient to receive a
169  response from the test FTP server
170- `%GOPHER6PORT` - IPv6 port number of the Gopher server
171- `%GOPHERPORT` - Port number of the Gopher server
172- `%GOPHERSPORT` - Port number of the Gophers server
173- `%HOST6IP` - IPv6 address of the host running this test
174- `%HOSTIP` - IPv4 address of the host running this test
175- `%HTTP2PORT` - Port number of the HTTP/2 server
176- `%HTTP6PORT` - IPv6 port number of the HTTP server
177- `%HTTPPORT` - Port number of the HTTP server
178- `%HTTPSPORT` - Port number of the HTTPS server
179- `%HTTPSPROXYPORT` - Port number of the HTTPS-proxy
180- `%HTTPTLS6PORT` - IPv6 port number of the HTTP TLS server
181- `%HTTPTLSPORT` - Port number of the HTTP TLS server
182- `%HTTPUNIXPATH` - Path to the Unix socket of the HTTP server
183- `%IMAP6PORT` - IPv6 port number of the IMAP server
184- `%IMAPPORT` - Port number of the IMAP server
185- `%LOGDIR` - Log directory relative to %PWD
186- `%MQTTPORT` - Port number of the MQTT server
187- `%MQTTSPORT` - Port number of the MQTTS server
188- `%NOLISTENPORT` - Port number where no service is listening
189- `%POP36PORT` - IPv6 port number of the POP3 server
190- `%POP3PORT` - Port number of the POP3 server
191- `%PROXYPORT` - Port number of the HTTP proxy
192- `%PWD` - Current directory
193- `%RESOLVE` - server/resolve command
194- `%RTSP6PORT` - IPv6 port number of the RTSP server
195- `%RTSPPORT` - Port number of the RTSP server
196- `%SMBPORT` - Port number of the SMB server
197- `%SMBSPORT` - Port number of the SMBS server
198- `%SMTP6PORT` - IPv6 port number of the SMTP server
199- `%SMTPPORT` - Port number of the SMTP server
200- `%SOCKSPORT` - Port number of the SOCKS4/5 server
201- `%SOCKSUNIXPATH` - Path to the Unix socket of the SOCKS server
202- `%SRCDIR` - Full path to the source dir
203- `%SCP_PWD` - Current directory friendly for the SSH server for the scp:// protocol
204- `%SFTP_PWD` - Current directory friendly for the SSH server for the sftp:// protocol
205- `%SSHKEYALGO` - SSH host and client key algorithm, e.g. `ssh-rsa` or `ssh-ed25519`
206- `%SSHPORT` - Port number of the SCP/SFTP server
207- `%SSHSRVMD5` - MD5 of SSH server's public key
208- `%SSHSRVSHA256` - SHA256 of SSH server's public key
209- `%TELNETPORT` - Port number of the telnet server
210- `%TESTNUMBER` - Number of the test case
211- `%TFTP6PORT` - IPv6 port number of the TFTP server
212- `%TFTPPORT` - Port number of the TFTP server
213- `%USER` - Login ID of the user running the test
214- `%VERNUM` - the version number of the tested curl (without -DEV)
215- `%VERSION` - the full version number of the tested curl
216
217# `<testcase>`
218
219Each test is always specified entirely within the `testcase` tag. Each test
220case is split up in four main sections: `info`, `reply`, `client` and
221`verify`.
222
223- **info** provides information about the test case
224
225- **reply** is used for the server to know what to send as a reply for the
226requests curl sends
227
228- **client** defines how the client should behave
229
230- **verify** defines how to verify that the data stored after a command has
231been run ended up correct
232
233Each main section has a number of available subsections that can be specified,
234that are checked/used if specified.
235
236## `<info>`
237
238### `<keywords>`
239A newline-separated list of keywords describing what this test case uses and
240tests. Try to use already used keywords. These keywords are used for
241statistical/informational purposes and for choosing or skipping classes of
242tests. Keywords must begin with an alphabetic character, `-`, `[` or `{` and
243may actually consist of multiple words separated by spaces which are treated
244together as a single identifier. Most keywords are only there to provide a way
245for users to skip certain classes of tests, if desired, but a few are treated
246specially by the test harness or build system.
247
248When running a unit test and the keywords include `unittest`, the `<tool>`
249section can be left empty to use the standard unit test tool name `unitN` where
250`N` is the test number.
251
252The `test-ci` make target automatically skips test with the `flaky` keyword.
253
254Tests that have strict timing dependencies have the `timing-dependent` keyword.
255These are intended to eventually be treated specially on CI builds which are
256often run on overloaded machines with unpredictable timing.
257
258Tests using non-7-bit-ASCII characters must provide them with `%hex[]` or
259similar.
260
261## `<reply>`
262
263### `<data [nocheck="yes"] [sendzero="yes"] [nonewline="yes"] [crlf="yes|headers"]>`
264
265data to be sent to the client on its request and later verified that it
266arrived safely. Set `nocheck="yes"` to prevent the test script from verifying
267the arrival of this data.
268
269If the data contains `swsclose` anywhere within the start and end tag, and
270this is an HTTP test, then the connection is closed by the server after this
271response is sent. If not, the connection is kept persistent.
272
273If the data contains `swsbounce` anywhere within the start and end tag, then
274the HTTP server overrides the part number response returned for a subsequent
275request made by the same test to `previous part number + 1`. For example, if a
276test makes a request which causes the server to return `<data>` that contains
277keyword `swsbounce` then for the next response it ignores the requested part
278number and instead returns `<data1>`. If `<data1>` contains keyword
279`swsbounce` then the next response is `<data2>` and so on. This is useful for
280auth tests and similar.
281
282`sendzero=yes` means that the (FTP) server "sends" the data even if the size
283is zero bytes. Used to verify curl's behavior on zero bytes transfers.
284
285`nonewline=yes` means that the last byte (the trailing newline character)
286should be cut off from the data before sending or comparing it.
287
288`crlf=yes` forces the newlines to become CRLF even if not written so in the
289test.
290
291`crlf=headers` forces *header* newlines to become CRLF even if not written so
292in the source file. Note that this makes runtests.pl parse and "guess" what is
293a header and what is not in order to apply the CRLF line endings appropriately.
294
295For FTP file listings, the `<data>` section is be used *only* if you make sure
296that there has been a CWD done first to a directory named `test-[NUM]` where
297`NUM` is the test case number. Otherwise the ftp server cannot know from which
298test file to load the list content.
299
300### `<dataNUM [crlf="yes|headers"]>`
301
302Send back this contents instead of the `<data>` one. The `NUM` is set by:
303
304- The test number in the request line is >10000 and this is the remainder of
305  [test case number]%10000.
306- The request was HTTP and included digest details, which adds 1000 to `NUM`
307- If an HTTP request is NTLM type-1, it adds 1001 to `NUM`
308- If an HTTP request is NTLM type-3, it adds 1002 to `NUM`
309- If an HTTP request is Basic and `NUM` is already >=1000, it adds 1 to `NUM`
310- If an HTTP request is Negotiate, `NUM` gets incremented by one for each
311  request with Negotiate authorization header on the same test case.
312
313Dynamically changing `NUM` in this way allows the test harness to be used to
314test authentication negotiation where several different requests must be sent
315to complete a transfer. The response to each request is found in its own data
316section. Validating the entire negotiation sequence can be done by specifying
317a `datacheck` section.
318
319### `<connect [crlf="yes|headers"]>`
320The connect section is used instead of the 'data' for all CONNECT
321requests. The remainder of the rules for the data section then apply but with
322a connect prefix.
323
324`crlf=yes` forces the newlines to become CRLF even if not written so in the
325test.
326
327`crlf=headers` forces *header* newlines to become CRLF even if not written so
328in the source file. Note that this makes runtests.pl parse and "guess" what is
329a header and what is not in order to apply the CRLF line endings appropriately.
330
331### `<socks>`
332Address type and address details as logged by the SOCKS proxy.
333
334### `<datacheck [mode="text"] [nonewline="yes"] [crlf="yes|headers"]>`
335if the data is sent but this is what should be checked afterwards. If
336`nonewline=yes` is set, runtests cuts off the trailing newline from the data
337before comparing with the one actually received by the client.
338
339Use the `mode="text"` attribute if the output is in text mode on platforms
340that have a text/binary difference.
341
342### `<datacheckNUM [nonewline="yes"] [mode="text"] [crlf="yes|headers"]>`
343The contents of numbered `datacheck` sections are appended to the non-numbered
344one.
345
346### `<size>`
347number to return on an ftp SIZE command (set to -1 to make this command fail)
348
349### `<mdtm>`
350what to send back if the client sends a (FTP) `MDTM` command, set to -1 to
351have it return that the file does not exist
352
353### `<postcmd>`
354special purpose server-command to control its behavior *after* the
355reply is sent
356For HTTP/HTTPS, these are supported:
357
358`wait [secs]` - Pause for the given time
359
360### `<servercmd>`
361Special-commands for the server.
362
363The first line of this file is always set to `Testnum [number]` by the test
364script, to allow servers to read that to know what test the client is about to
365issue.
366
367#### For FTP/SMTP/POP/IMAP
368
369- `REPLY [command] [return value] [response string]` - Changes how the server
370  responds to the [command]. [response string] is evaluated as a perl string,
371  so it can contain embedded \r\n, for example. There is a special [command]
372  named "welcome" (without quotes) which is the string sent immediately on
373  connect as a welcome.
374- `REPLYLF` (like above but sends the response terminated with LF-only and not
375   CRLF)
376- `COUNT [command] [num]` - Do the `REPLY` change for `[command]` only `[num]`
377  times and then go back to the built-in approach
378- `DELAY [command] [secs]` - Delay responding to this command for the given
379  time
380- `RETRWEIRDO` - Enable the "weirdo" RETR case when multiple response lines
381   appear at once when a file is transferred
382- `RETRNOSIZE` - Make sure the RETR response does not contain the size of the
383  file
384- `RETRSIZE [size]` - Force RETR response to contain the specified size
385- `NOSAVE` - Do not actually save what is received
386- `SLOWDOWN` - Send FTP responses with 0.01 sec delay between each byte
387- `SLOWDOWNDATA` - Send FTP responses with 0.01 sec delay between each data
388  byte
389- `PASVBADIP` - makes PASV send back an illegal IP in its 227 response
390- `CAPA [capabilities]` - Enables support for and specifies a list of space
391   separated capabilities to return to the client for the IMAP `CAPABILITY`,
392   POP3 `CAPA` and SMTP `EHLO` commands
393- `AUTH [mechanisms]` - Enables support for SASL authentication and specifies
394   a list of space separated mechanisms for IMAP, POP3 and SMTP
395- `STOR [msg]` respond with this instead of default after `STOR`
396
397#### For HTTP/HTTPS
398
399- `auth_required` if this is set and a POST/PUT is made without auth, the
400  server does NOT wait for the full request body to get sent
401- `delay: [msecs]` - delay this amount after connection
402- `idle` - do nothing after receiving the request, "sit idle"
403- `stream` - continuously send data to the client, never-ending
404- `writedelay: [msecs]` delay this amount between reply packets
405- `skip: [num]` - instructs the server to ignore reading this many bytes from
406  a PUT or POST request
407- `rtp: part [num] channel [num] size [num]` - stream a fake RTP packet for
408  the given part on a chosen channel with the given payload size
409- `connection-monitor` - When used, this logs `[DISCONNECT]` to the
410  `server.input` log when the connection is disconnected.
411- `upgrade` - when an HTTP upgrade header is found, the server upgrades to
412  http2
413- `swsclose` - instruct server to close connection after response
414- `no-expect` - do not read the request body if Expect: is present
415
416#### For TFTP
417`writedelay: [secs]` delay this amount between reply packets (each packet
418  being 512 bytes payload)
419
420### `<dns>`
421
422Commands for the test DNS server.
423
424- `A: [dotted ipv4 address]` - set IPv4 address to return
425- `AAAA: [numerical IPv6 address]` - set IPv6 address to return, with or
426  without `[]`
427
428## `<client>`
429
430### `<server>`
431What server(s) this test case requires/uses. Available servers:
432
433- `dict`
434- `file`
435- `ftp`
436- `ftp-ipv6`
437- `ftps`
438- `gopher`
439- `gopher-ipv6`
440- `gophers`
441- `http`
442- `http/2`
443- `http-ipv6`
444- `http-proxy`
445- `https`
446- `https-proxy`
447- `https-mtls`
448- `httptls+srp`
449- `httptls+srp-ipv6`
450- `http-unix`
451- `imap`
452- `mqtt`
453- `pop3`
454- `rtsp`
455- `rtsp-ipv6`
456- `scp`
457- `sftp`
458- `smb`
459- `smtp`
460- `socks4`
461- `socks5`
462- `socks5unix`
463- `telnet`
464- `tftp`
465
466Give only one per line. If a test does not require any servers, the `<server>`
467subsection should be omitted.
468
469Servers that require a special server certificate can
470have the PEM certificate filename (found in the `certs` directory) appended to
471the server name separated by a space.
472
473### `<features>`
474A list of features that MUST be present in the client/library for this test to
475be able to run. If a required feature is not present then the test is SKIPPED.
476
477Alternatively a feature can be prefixed with an exclamation mark to indicate a
478feature is NOT required. If the feature is present then the test is SKIPPED.
479
480Features testable here are:
481
482- `--libcurl`
483- `alt-svc`
484- `aws` - built with **aws-sigv4** support
485- `AppleIDN`
486- `asyn-rr` - c-ares is used for additional records only
487- `brotli`
488- `c-ares` - c-ares is used for (all) name resolves
489- `CharConv`
490- `codeset-utf8`. If the running codeset is UTF-8 capable.
491- `cookies`
492- `crypto`
493- `cygwin`
494- `Debug`
495- `digest`
496- `DoH`
497- `getrlimit`
498- `GnuTLS`
499- `GSS-API`
500- `h2c`
501- `headers-api`
502- `HSTS`
503- `HTTP-auth`
504- `http/2`
505- `http/3`
506- `HTTPS-proxy`
507- `HTTPSRR`
508- `IDN`
509- `IPv6`
510- `Kerberos`
511- `Largefile`
512- `large-time` (time_t is larger than 32-bit)
513- `large-size` (size_t is larger than 32-bit)
514- `libssh2`
515- `libssh`
516- `badlibssh` (libssh configuration incompatible with the test suite)
517- `libz`
518- `local-http`. The HTTP server runs on 127.0.0.1
519- `manual`
520- `mbedtls`
521- `Mime`
522- `netrc`
523- `nghttpx`
524- `nghttpx-h3`
525- `NTLM`
526- `NTLM_WB`
527- `OpenSSL`
528- `override-dns` - this build can use a "fake" DNS server
529- `parsedate`
530- `proxy`
531- `PSL`
532- `rustls`
533- `Schannel`
534- `shuffle-dns`
535- `socks`
536- `SPNEGO`
537- `SSL`
538- `SSLpinning`
539- `SSPI`
540- `threaded-resolver`
541- `TLS-SRP`
542- `TrackMemory`
543- `typecheck`
544- `threadsafe`
545- `Unicode`
546- `unittest`
547- `UnixSockets`
548- `verbose-strings`
549- `wakeup`
550- `win32`
551- `WinIDN`
552- `wolfssl`
553- `xattr`
554- `zstd`
555
556as well as each protocol that curl supports. A protocol only needs to be
557specified if it is different from the server (useful when the server is
558`none`).
559
560### `<killserver>`
561Using the same syntax as in `<server>` but when mentioned here these servers
562are explicitly KILLED when this test case is completed. Only use this if there
563is no other alternatives. Using this of course requires subsequent tests to
564restart servers.
565
566### `<precheck>`
567A command line that if set gets run by the test script before the test. If an
568output is displayed by the command or if the return code is non-zero, the test
569is skipped and the (single-line) output is displayed as reason for not running
570the test.
571
572### `<tool>`
573Name of tool to invoke instead of "curl". This tool must be built and exist
574either in the `libtest/` directory (if the tool name starts with `lib`) or in
575the `unit/` directory (if the tool name starts with `unit`).
576
577### `<name>`
578Brief test case description, shown when the test runs.
579
580### `<setenv>`
581
582    variable1=contents1
583    variable2=contents2
584    variable3
585
586Set the given environment variables to the specified value before the actual
587command is run. They are restored back to their former values again after the
588command has been run.
589
590If the variable name has no assignment, no `=`, then that variable is deleted.
591
592### `<command [option="no-q/no-output/no-include/no-memdebug/force-output/binary-trace"] [timeout="secs"][delay="secs"][type="perl/shell"]>`
593Command line to run.
594
595If the command spans multiple lines, they are concatenated with a space added
596between them.
597
598Set `type="perl"` to write the test case as a perl script. It implies that
599there is no memory debugging and valgrind gets shut off for this test.
600
601Set `type="shell"` to write the test case as a shell script. It implies that
602there is no memory debugging and valgrind gets shut off for this test.
603
604Set `option="no-output"` to prevent the test script to slap on the `--output`
605argument that directs the output to a file. The `--output` is also not added
606if the verify/stdout section is used.
607
608Set `option="force-output"` to make use of `--output` even when the test is
609otherwise written to verify stdout.
610
611Set `option="no-include"` to prevent the test script to slap on the
612`--include` argument.
613
614Set `option="no-memdebug"` to make the test run without the memdebug tracking
615system. For tests that are incompatible - multi-threaded for example.
616
617Set `option="no-q"` avoid using `-q` as the first argument in the curl command
618line.
619
620Set `option="binary-trace"` to use `--trace` instead of `--trace-ascii` for
621tracing. Suitable for binary-oriented protocols such as MQTT.
622
623Set `timeout="secs"` to override default server logs advisor read lock
624timeout. This timeout is used by the test harness, once that the command has
625completed execution, to wait for the test server to write out server side log
626files and remove the lock that advised not to read them. The "secs" parameter
627is the not negative integer number of seconds for the timeout. This `timeout`
628attribute is documented for completeness sake, but is deep test harness stuff
629and only needed for singular and specific test cases. Avoid using it.
630
631Set `delay="secs"` to introduce a time delay once that the command has
632completed execution and before the `<postcheck>` section runs. The "secs"
633parameter is the not negative integer number of seconds for the delay. This
634'delay' attribute is intended for specific test cases, and normally not
635needed.
636
637### `<file name="%LOGDIR/filename" [options]>`
638This creates the named file with this content before the test case is run,
639which is useful if the test case needs a file to act on.
640
641If `nonewline="yes"` is used, the created file gets the final newline stripped
642off.
643
644`crlf=yes` forces the newlines to become CRLF even if not written so in the
645test.
646
647`mode="text"` normalizes the line endings to make them compare as text on all
648platforms.
649
650### `<file1>`
6511 to 4 can be appended to 'file' to create more files.
652
653### `<file2>`
654
655### `<file3>`
656
657### `<file4>`
658
659### `<stdin [nonewline="yes"][crlf="yes"]>`
660Pass this given data on stdin to the tool.
661
662If `nonewline` is set, we cut off the trailing newline of this given data
663before comparing with the one actually received by the client
664
665`crlf=yes` forces the newlines to become CRLF even if not written so in the
666test.
667
668## `<disable>`
669
670If `test-duphandle` is a listed item here, this is not run when
671`--test-duphandle` is used.
672
673## `<verify>`
674### `<errorcode>`
675numerical error code curl is supposed to return. Specify a list of accepted
676error codes by separating multiple numbers with comma. See test 237 for an
677example.
678
679### `<strip>`
680One regex per line that is removed from the protocol dumps before the
681comparison is made. This is useful to remove dependencies on dynamically
682changing protocol data such as port numbers or user-agent strings.
683
684### `<strippart>`
685One perl op per line that operates on the protocol dump. This is pretty
686advanced. Example: `s/^EPRT .*/EPRT stripped/`.
687
688### `<postcheck>`
689A command line that if set gets run by the test script after the test. If the
690command exists with a non-zero status code, the test is considered failed.
691
692### `<notexists>`
693A list of directory entries that are checked for after the test has completed
694and that must not exist. A listed entry existing causes the test to fail.
695
696### `<protocol [nonewline="yes"][crlf="yes|headers"]>`
697
698the protocol dump curl should transmit, if `nonewline` is set, we cut off the
699trailing newline of this given data before comparing with the one actually
700sent by the client The `<strip>` and `<strippart>` rules are applied before
701comparisons are made.
702
703`crlf=yes` forces the newlines to become CRLF even if not written so in the
704test.
705
706`crlf=headers` forces *header* newlines to become CRLF even if not written so
707in the source file. Note that this makes runtests.pl parse and "guess" what is
708a header and what is not in order to apply the CRLF line endings appropriately.
709
710### `<proxy [nonewline="yes"][crlf="yes|headers"]>`
711
712The protocol dump curl should transmit to an HTTP proxy (when the http-proxy
713server is used), if `nonewline` is set, we cut off the trailing newline of
714this given data before comparing with the one actually sent by the client The
715`<strip>` and `<strippart>` rules are applied before comparisons are made.
716
717### `<stderr [mode="text/warn"] [nonewline="yes"] [crlf="yes|headers"]>`
718This verifies that this data was passed to stderr.
719
720Use the mode="text" attribute if the output is in text mode on platforms that
721have a text/binary difference.
722
723Use the mode="warn" attribute for curl warning output, as it then makes the
724check without newlines and the prefix to better handle that the lines may wrap
725at different points depending on the lengths of the lines and terminal width.
726
727`crlf=yes` forces the newlines to become CRLF even if not written so in the
728test.
729
730`crlf=headers` forces *header* newlines to become CRLF even if not written so
731in the source file. Note that this makes runtests.pl parse and "guess" what is
732a header and what is not in order to apply the CRLF line endings appropriately.
733
734If `nonewline` is set, we cut off the trailing newline of this given data
735before comparing with the one actually received by the client
736
737### `<stdout [mode="text"] [nonewline="yes"] [crlf="yes|headers"] [loadfile="filename"]>`
738This verifies that this data was passed to stdout.
739
740Use the mode="text" attribute if the output is in text mode on platforms that
741have a text/binary difference.
742
743If `nonewline` is set, we cut off the trailing newline of this given data
744before comparing with the one actually received by the client
745
746`crlf=yes` forces the newlines to become CRLF even if not written so in the
747test.
748
749`crlf=headers` forces *header* newlines to become CRLF even if not written so
750in the source file. Note that this makes runtests.pl parse and "guess" what is
751a header and what is not in order to apply the CRLF line endings appropriately.
752
753`loadfile="filename"` makes loading the data from an external file.
754
755To verify that there was nothing sent to stdout, put `%EMPTY` as the only
756content.
757
758### `<limit>`
759
760When this test runs and curl was built with debug enabled, runtests make sure
761that the set limits are not exceeded. Supported limits:
762
763    Allocations: [number of allocation calls]
764    Maximum allocated: [maximum concurrent memory allocated]
765
766### `<file name="%LOGDIR/filename" [mode="text"] [crlf="yes|headers"]>`
767The file's contents must be identical to this after the test is complete. Use
768the mode="text" attribute if the output is in text mode on platforms that have
769a text/binary difference.
770
771`crlf=yes` forces the newlines to become CRLF even if not written so in the
772test.
773
774`crlf=headers` forces *header* newlines to become CRLF even if not written so
775in the source file. Note that this makes runtests.pl parse and "guess" what is
776a header and what is not in order to apply the CRLF line endings appropriately.
777
778### `<file1>`
7791 to 4 can be appended to 'file' to compare more files.
780
781### `<file2>`
782
783### `<file3>`
784
785### `<file4>`
786
787### `<stripfile>`
788One perl op per line that operates on the output file or stdout before being
789compared with what is stored in the test file. This is pretty
790advanced. Example: "s/^EPRT .*/EPRT stripped/"
791
792### `<stripfile1>`
7931 to 4 can be appended to `stripfile` to strip the corresponding `<fileN>`
794content
795
796### `<stripfile2>`
797
798### `<stripfile3>`
799
800### `<stripfile4>`
801
802### `<upload [crlf="yes"] [nonewline="yes"]>`
803the contents of the upload data curl should have sent
804
805`crlf=yes` forces *upload* newlines to become CRLF even if not written so in
806the source file.
807
808`nonewline=yes` means that the last byte (the trailing newline character)
809should be cut off from the upload data before comparing it.
810
811### `<valgrind>`
812disable - disables the valgrind log check for this test
813
814### `<dns [host="name"]>`
815
816This specify the input the DNS server is expected to get from curl. Because of
817differences in implementations, this section is sorted automatically before
818compared.
819
820Because of local configurations in machines running tests, there may be
821additional requests sent to `[host].[custom suffix]`. To prevent such requests
822to mess up comparisons, we can set the hostname to check in the `<dns>` tag.