luna - curl/docs/TODO.md

   1<!--
   2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
   3
   4SPDX-License-Identifier: curl
   5-->
   6
   7# TODO intro
   8
   9Things to do in project curl. Please tell us what you think, contribute and
  10send us patches that improve things.
  11
  12Be aware that these are things that we could do, or have once been considered
  13things we could do. If you want to work on any of these areas, please consider
  14bringing it up for discussions first on the mailing list so that we all agree
  15it is still a good idea for the project.
  16
  17All bugs documented in the [known_bugs
  18document](https://curl.se/docs/knownbugs.html) are subject for fixing.
  19
  20# libcurl
  21
  22## Consult `%APPDATA%` also for `.netrc`
  23
  24`%APPDATA%\.netrc` is not considered when running on Windows. Should it not?
  25
  26See [curl issue 4016](https://github.com/curl/curl/issues/4016)
  27
  28## `struct lifreq`
  29
  30Use `struct lifreq` and `SIOCGLIFADDR` instead of `struct ifreq` and
  31`SIOCGIFADDR` on newer Solaris versions as they claim the latter is obsolete.
  32To support IPv6 interface addresses for network interfaces properly.
  33
  34## alt-svc sharing
  35
  36The share interface could benefit from allowing the alt-svc cache to be
  37possible to share between easy handles.
  38
  39See [curl issue 4476](https://github.com/curl/curl/issues/4476)
  40
  41The share interface offers CURL_LOCK_DATA_CONNECT to have multiple easy
  42handle share a connection cache, but due to how connections are used they are
  43still not thread-safe when used shared.
  44
  45See [curl issue 4915](https://github.com/curl/curl/issues/4915) and lib1541.c
  46
  47The share interface offers CURL_LOCK_DATA_HSTS to have multiple easy handle
  48share an HSTS cache, but this is not thread-safe.
  49
  50## thread-safe sharing
  51
  52Using the share interface users can share some data between easy handles but
  53several of the sharing options are documented as not safe and supported to
  54share between multiple concurrent threads. Fixing this would enable more users
  55to share data in more powerful ways.
  56
  57## updated DNS server while running
  58
  59If `/etc/resolv.conf` gets updated while a program using libcurl is running, it
  60is may cause name resolves to fail unless `res_init()` is called. We should
  61consider calling `res_init()` + retry once unconditionally on all name resolve
  62failures to mitigate against this. Firefox works like that. Note that Windows
  63does not have `res_init()` or an alternative.
  64
  65[curl issue 2251](https://github.com/curl/curl/issues/2251)
  66
  67## c-ares and CURLOPT_OPENSOCKETFUNCTION
  68
  69curl creates most sockets via the CURLOPT_OPENSOCKETFUNCTION callback and
  70close them with the CURLOPT_CLOSESOCKETFUNCTION callback. c-ares does not use
  71those functions and instead opens and closes the sockets itself. This means
  72that when curl passes the c-ares socket to the CURLMOPT_SOCKETFUNCTION it is
  73not owned by the application like other sockets.
  74
  75See [curl issue 2734](https://github.com/curl/curl/issues/2734)
  76
  77## Monitor connections in the connection pool
  78
  79libcurl's connection cache or pool holds a number of open connections for the
  80purpose of possible subsequent connection reuse. It may contain a few up to a
  81significant amount of connections. Currently, libcurl leaves all connections
  82as they are and first when a connection is iterated over for matching or reuse
  83purpose it is verified that it is still alive.
  84
  85Those connections may get closed by the server side for idleness or they may
  86get an HTTP/2 ping from the peer to verify that they are still alive. By
  87adding monitoring of the connections while in the pool, libcurl can detect
  88dead connections (and close them) better and earlier, and it can handle HTTP/2
  89pings to keep such ones alive even when not actively doing transfers on them.
  90
  91## Try to URL encode given URL
  92
  93Given a URL that for example contains spaces, libcurl could have an option
  94that would try somewhat harder than it does now and convert spaces to %20 and
  95perhaps URL encoded byte values over 128 etc (do what the redirect following
  96code already does).
  97
  98[curl issue 514](https://github.com/curl/curl/issues/514)
  99
 100## Add support for IRIs
 101
 102IRIs (RFC 3987) allow localized, non-ASCII, names in the URL. To properly
 103support this, curl/libcurl would need to translate/encode the given input
 104from the input string encoding into percent encoded output "over the wire".
 105
 106To make that work smoothly for curl users even on Windows, curl would probably
 107need to be able to convert from several input encodings.
 108
 109## try next proxy if one does not work
 110
 111Allow an application to specify a list of proxies to try, and failing to
 112connect to the first go on and try the next instead until the list is
 113exhausted. Browsers support this feature at least when they specify proxies
 114using `PAC`.
 115
 116[curl issue 896](https://github.com/curl/curl/issues/896)
 117
 118## provide timing info for each redirect
 119
 120curl and libcurl provide timing information via a set of different time-stamps
 121(CURLINFO_*_TIME). When curl is following redirects, those returned time value
 122are the accumulated sums. An improvement could be to offer separate timings
 123for each redirect.
 124
 125[curl issue 6743](https://github.com/curl/curl/issues/6743)
 126
 127## CURLINFO_PAUSE_STATE
 128
 129Return information about the transfer's current pause state, in both
 130directions. See [curl issue 2588](https://github.com/curl/curl/issues/2588)
 131
 132## Expose tried IP addresses that failed
 133
 134When libcurl fails to connect to a host, it could offer the application the
 135addresses that were used in the attempt. Source + destination IP, source +
 136destination port and protocol (UDP or TCP) for each failure. Possibly as a
 137callback. Perhaps also provide reason.
 138
 139[curl issue 2126](https://github.com/curl/curl/issues/2126)
 140
 141## erase secrets from heap/stack after use
 142
 143Introducing a concept and system to erase secrets from memory after use, it
 144could help mitigate and lessen the impact of (future) security problems etc.
 145However: most secrets are passed to libcurl as clear text from the application
 146and then clearing them within the library adds nothing...
 147
 148[curl issue 7268](https://github.com/curl/curl/issues/7268)
 149
 150## make DoH inherit more transfer properties
 151
 152Some options are not inherited because they are not relevant for the DoH SSL
 153connections, or inheriting the option may result in unexpected behavior. For
 154example the user's debug function callback is not inherited because it would
 155be unexpected for internal handles (i.e DoH handles) to be passed to that
 156callback.
 157
 158If an option is not inherited then it is not possible to set it separately
 159for DoH without a DoH-specific option. For example:
 160`CURLOPT_DOH_SSL_VERIFYHOST`, `CURLOPT_DOH_SSL_VERIFYPEER` and
 161`CURLOPT_DOH_SSL_VERIFYSTATUS`.
 162
 163See [curl issue 6605](https://github.com/curl/curl/issues/6605)
 164
 165# libcurl - multi interface
 166
 167## More non-blocking
 168
 169Make sure we do not ever loop because of non-blocking sockets returning
 170`EWOULDBLOCK` or similar. Blocking cases include:
 171
 172- Name resolves on non-Windows unless c-ares or the threaded resolver is used.
 173- The threaded resolver may block on cleanup:
 174  [curl issue 4852](https://github.com/curl/curl/issues/4852)
 175- `file://` transfers
 176- TELNET transfers
 177- GSSAPI authentication for FTP transfers
 178- The "DONE" operation (post transfer protocol-specific actions) for the
 179protocols SFTP, SMTP, FTP. Fixing `multi_done()` for this is a worthy task.
 180- `curl_multi_remove_handle()` for any of the above.
 181- Calling `curl_ws_send()` from a callback
 182
 183## Better support for same name resolves
 184
 185If a name resolve has been initiated for a given name and a second easy handle
 186wants to resolve that same name as well, make it wait for the first resolve to
 187end up in the cache instead of doing a second separate resolve. This is
 188especially needed when adding many simultaneous handles using the same
 189hostname when the DNS resolver can get flooded.
 190
 191## Non-blocking `curl_multi_remove_handle()`
 192
 193The multi interface has a few API calls that assume a blocking behavior, like
 194`add_handle()` and `remove_handle()` which limits what we can do internally.
 195The multi API need to be moved even more into a single function that "drives"
 196everything in a non-blocking manner and signals when something is done. A
 197remove or add would then only ask for the action to get started and then
 198`multi_perform()` etc still be called until the add/remove is completed.
 199
 200## Split connect and authentication process
 201
 202The multi interface treats the authentication process as part of the connect
 203phase. As such any failures during authentication does not trigger the
 204relevant QUIT or LOGOFF for protocols such as IMAP, POP3 and SMTP.
 205
 206## Edge-triggered sockets should work
 207
 208The multi_socket API should work with edge-triggered socket events. One of the
 209internal actions that need to be improved for this to work perfectly is the
 210`maxloops` handling in `transfer.c:readwrite_data()`.
 211
 212## multi upkeep
 213
 214In libcurl 7.62.0 we introduced `curl_easy_upkeep`. It unfortunately only
 215works on easy handles. We should introduces a version of that for the multi
 216handle, and also consider doing `upkeep` automatically on connections in the
 217connection pool when the multi handle is in used.
 218
 219See [curl issue 3199](https://github.com/curl/curl/issues/3199)
 220
 221## Virtual external sockets
 222
 223libcurl performs operations on the given file descriptor that presumes it is a
 224socket and an application cannot replace them at the moment. Allowing an
 225application to fully replace those would allow a larger degree of freedom and
 226flexibility.
 227
 228See [curl issue 5835](https://github.com/curl/curl/issues/5835)
 229
 230## dynamically decide to use socketpair
 231
 232For users who do not use `curl_multi_wait()` or do not care for
 233`curl_multi_wakeup()`, we could introduce a way to make libcurl NOT create a
 234socketpair in the multi handle.
 235
 236See [curl issue 4829](https://github.com/curl/curl/issues/4829)
 237
 238# Documentation
 239
 240## Improve documentation about fork safety
 241
 242See [curl issue 6968](https://github.com/curl/curl/issues/6968)
 243
 244# FTP
 245
 246## A fixed directory listing format
 247
 248Since listing the contents of a remove directory with FTP is returning the
 249list in a format and style the server likes without any established or even
 250defacto standard existing, it would be a feature to users if curl could parse
 251the directory listing and output a general curl format that is fixed and the
 252same, independent of the server's choice. This would allow users to better and
 253more reliably extract information about remote content via FTP directory
 254listings.
 255
 256## GSSAPI via Windows SSPI
 257
 258In addition to currently supporting the SASL GSSAPI mechanism (Kerberos V5)
 259via third-party GSS-API libraries, such as MIT Kerberos, also add support for
 260GSSAPI authentication via Windows SSPI.
 261
 262## STAT for LIST without data connection
 263
 264Some FTP servers allow STAT for listing directories instead of using LIST, and
 265the response is then sent over the control connection instead of as the
 266otherwise used data connection.
 267
 268This is not detailed in any FTP specification.
 269
 270## Passive transfer could try other IP addresses
 271
 272When doing FTP operations through a proxy at localhost, the reported spotted
 273that curl only tried to connect once to the proxy, while it had multiple
 274addresses and a failed connect on one address should make it try the next.
 275
 276After switching to passive mode (EPSV), curl could try all IP addresses for
 277`localhost`. Currently it tries `::1`, but it should also try `127.0.0.1`.
 278
 279See [curl issue 1508](https://github.com/curl/curl/issues/1508)
 280
 281# HTTP
 282
 283## Provide the error body from a CONNECT response
 284
 285When curl receives a body response from a CONNECT request to a proxy, it
 286always reads and ignores it. It would make some users happy if curl instead
 287optionally would be able to make that responsible available. Via a new
 288callback? Through some other means?
 289
 290See [curl issue 9513](https://github.com/curl/curl/issues/9513)
 291
 292## Obey `Retry-After` in redirects
 293
 294The `Retry-After` response header is said to dictate "the minimum time that
 295the user agent is asked to wait before issuing the redirected request" and
 296libcurl does not obey this.
 297
 298See [curl issue 11447](https://github.com/curl/curl/issues/11447)
 299
 300## Rearrange request header order
 301
 302Server implementers often make an effort to detect browser and to reject
 303clients it can detect to not match. One of the last details we cannot yet
 304control in libcurl's HTTP requests, which also can be exploited to detect that
 305libcurl is in fact used even when it tries to impersonate a browser, is the
 306order of the request headers. I propose that we introduce a new option in
 307which you give headers a value, and then when the HTTP request is built it
 308sorts the headers based on that number. We could then have internally created
 309headers use a default value so only headers that need to be moved have to be
 310specified.
 311
 312## Allow SAN names in HTTP/2 server push
 313
 314curl only allows HTTP/2 push promise if the provided :authority header value
 315exactly matches the hostname given in the URL. It could be extended to allow
 316any name that would match the Subject Alternative Names in the server's TLS
 317certificate.
 318
 319See [curl pull request 3581](https://github.com/curl/curl/pull/3581)
 320
 321## `auth=` in URLs
 322
 323Add the ability to specify the preferred authentication mechanism to use by
 324using `;auth=<mech>` in the login part of the URL.
 325
 326For example:
 327
 328`http://test:pass;auth=NTLM@example.com` would be equivalent to specifying
 329`--user test:pass;auth=NTLM` or `--user test:pass --ntlm` from the command
 330line.
 331
 332Additionally this should be implemented for proxy base URLs as well.
 333
 334## Require HTTP version X or higher
 335
 336curl and libcurl provide options for trying higher HTTP versions (for example
 337HTTP/2) but then still allows the server to pick version 1.1. We could
 338consider adding a way to require a minimum version.
 339
 340See [curl issue 7980](https://github.com/curl/curl/issues/7980)
 341
 342# TELNET
 343
 344## ditch stdin
 345
 346Reading input (to send to the remote server) on stdin is a crappy solution for
 347library purposes. We need to invent a good way for the application to be able
 348to provide the data to send.
 349
 350## ditch telnet-specific select
 351
 352Move the telnet support's network `select()` loop go away and merge the code
 353into the main transfer loop. Until this is done, the multi interface does not
 354work for telnet.
 355
 356## feature negotiation debug data
 357
 358Add telnet feature negotiation data to the debug callback as header data.
 359
 360## exit immediately upon connection if stdin is /dev/null
 361
 362If it did, curl could be used to probe if there is an server there listening
 363on a specific port. That is, the following command would exit immediately
 364after the connection is established with exit code 0:
 365
 366    curl -s --connect-timeout 2 telnet://example.com:80 </dev/null
 367
 368# SMTP
 369
 370## Pass NOTIFY option to CURLOPT_MAIL_RCPT
 371
 372Is there a way to pass the NOTIFY option to the CURLOPT_MAIL_RCPT option ? I
 373set a string that already contains a bracket. For instance something like
 374that: `curl_slist_append(recipients, "<foo@bar> NOTIFY=SUCCESS,FAILURE");`.
 375
 376[curl issue 8232](https://github.com/curl/curl/issues/8232)
 377
 378## Enhanced capability support
 379
 380Add the ability, for an application that uses libcurl, to obtain the list of
 381capabilities returned from the EHLO command.
 382
 383## Add `CURLOPT_MAIL_CLIENT` option
 384
 385Rather than use the URL to specify the mail client string to present in the
 386`HELO` and `EHLO` commands, libcurl should support a new `CURLOPT`
 387specifically for specifying this data as the URL is non-standard and to be
 388honest a bit of a hack.
 389
 390Please see the following thread for more information:
 391https://curl.se/mail/lib-2012-05/0178.html
 392
 393# POP3
 394
 395## Enhanced capability support
 396
 397Add the ability, for an application that uses libcurl, to obtain the list of
 398capabilities returned from the CAPA command.
 399
 400# IMAP
 401
 402## Enhanced capability support
 403
 404Add the ability, for an application that uses libcurl, to obtain the list of
 405capabilities returned from the CAPABILITY command.
 406
 407# LDAP
 408
 409## SASL based authentication mechanisms
 410
 411Currently the LDAP module only supports `ldap_simple_bind_s()` in order to
 412bind to an LDAP server. This function sends username and password details
 413using the simple authentication mechanism (as clear text). It should be
 414possible to use `ldap_bind_s()` instead specifying the security context
 415information ourselves.
 416
 417## `CURLOPT_SSL_CTX_FUNCTION` for LDAPS
 418
 419`CURLOPT_SSL_CTX_FUNCTION` works perfectly for HTTPS and email protocols, but
 420it has no effect for LDAPS connections.
 421
 422[curl issue 4108](https://github.com/curl/curl/issues/4108)
 423
 424## Paged searches on LDAP server
 425
 426[curl issue 4452](https://github.com/curl/curl/issues/4452)
 427
 428## Certificate-Based Authentication
 429
 430LDAPS not possible with macOS and Windows with Certificate-Based Authentication
 431
 432[curl issue 9641](https://github.com/curl/curl/issues/9641)
 433
 434# SMB
 435
 436## Support modern versions
 437
 438curl only supports version 1, which barely anyone is using anymore.
 439
 440## File listing support
 441
 442Add support for listing the contents of an SMB share. The output should
 443probably be the same as/similar to FTP.
 444
 445## Honor file timestamps
 446
 447The timestamp of the transferred file should reflect that of the original
 448file.
 449
 450## Use NTLMv2
 451
 452Currently the SMB authentication uses NTLMv1.
 453
 454## Create remote directories
 455
 456Support for creating remote directories when uploading a file to a directory
 457that does not exist on the server, like `--ftp-create-dirs`.
 458
 459# FILE
 460
 461## Directory listing on non-POSIX
 462
 463Listing the contents of a directory accessed with FILE only works on platforms
 464with `opendir()`. Support could be added for more systems, like Windows.
 465
 466# TLS
 467
 468## `TLS-PSK` with OpenSSL
 469
 470Transport Layer Security pre-shared key cipher suites (`TLS-PSK`) is a set of
 471cryptographic protocols that provide secure communication based on pre-shared
 472keys (`PSK`). These pre-shared keys are symmetric keys shared in advance among
 473the communicating parties.
 474
 475[curl issue 5081](https://github.com/curl/curl/issues/5081)
 476
 477## TLS channel binding
 478
 479TLS 1.2 and 1.3 provide the ability to extract some secret data from the TLS
 480connection and use it in the client request (usually in some sort of
 481authentication) to ensure that the data sent is bound to the specific TLS
 482connection and cannot be successfully intercepted by a proxy. This
 483functionality can be used in a standard authentication mechanism such as
 484GSS-API or SCRAM, or in custom approaches like custom HTTP Authentication
 485headers.
 486
 487For TLS 1.2, the binding type is usually `tls-unique`, and for TLS 1.3 it is
 488`tls-exporter`.
 489
 490- https://datatracker.ietf.org/doc/html/rfc5929
 491- https://datatracker.ietf.org/doc/html/rfc9266
 492- [curl issue 9226](https://github.com/curl/curl/issues/9226)
 493
 494## Defeat TLS fingerprinting
 495
 496By changing the order of TLS extensions provided in the TLS handshake, it is
 497sometimes possible to circumvent TLS fingerprinting by servers. The TLS
 498extension order is of course not the only way to fingerprint a client.
 499
 500## Consider OCSP stapling by default
 501
 502Treat a negative response a reason for aborting the connection. Since OCSP
 503stapling is presumed to get used much less in the future when Let's Encrypt
 504drops the OCSP support, the benefit of this might however be limited.
 505
 506[curl issue 15483](https://github.com/curl/curl/issues/15483)
 507
 508## Provide callback for cert verification
 509
 510OpenSSL supports a callback for customized verification of the peer
 511certificate, but this does not seem to be exposed in the libcurl APIs. Could
 512it be? There is so much that could be done if it were.
 513
 514## Less memory massaging with Schannel
 515
 516The Schannel backend does a lot of custom memory management we would rather
 517avoid: the repeated allocation + free in sends and the custom memory + realloc
 518system for encrypted and decrypted data. That should be avoided and reduced
 519for 1) efficiency and 2) safety.
 520
 521## Support DANE
 522
 523[DNS-Based Authentication of Named Entities
 524(DANE)](https://datatracker.ietf.org/doc/html/rfc6698) is a way to provide
 525SSL keys and certs over DNS using DNSSEC as an alternative to the CA model.
 526
 527A patch was posted on March 7 2013
 528(https://curl.se/mail/lib-2013-03/0075.html) but it was a too simple approach.
 529See Daniel's comments: https://curl.se/mail/lib-2013-03/0103.html
 530
 531Björn Stenberg once wrote a separate initial take on DANE that was never
 532completed.
 533
 534## TLS record padding
 535
 536TLS (1.3) offers optional record padding and OpenSSL provides an API for it. I
 537could make sense for libcurl to offer this ability to applications to make
 538traffic patterns harder to figure out by network traffic observers.
 539
 540See [curl issue 5398](https://github.com/curl/curl/issues/5398)
 541
 542## Support Authority Information Access certificate extension (AIA)
 543
 544AIA can provide various things like certificate revocation lists but more
 545importantly information about intermediate CA certificates that can allow
 546validation path to be fulfilled when the HTTPS server does not itself provide
 547them.
 548
 549Since AIA is about downloading certs on demand to complete a TLS handshake, it
 550is probably a bit tricky to get done right and a serious privacy leak.
 551
 552See [curl issue 2793](https://github.com/curl/curl/issues/2793)
 553
 554## Some TLS options are not offered for HTTPS proxies
 555
 556Some TLS related options to the command line tool and libcurl are only
 557provided for the server and not for HTTPS proxies. `--proxy-tls-max`,
 558`--proxy-tlsv1.3`, `--proxy-curves` and a few more. For more Documentation on
 559this see: https://curl.se/libcurl/c/tls-options.html
 560
 561[curl issue 12286](https://github.com/curl/curl/issues/12286)
 562
 563## Make sure we forbid TLS 1.3 post-handshake authentication
 564
 565RFC 8740 explains how using HTTP/2 must forbid the use of TLS 1.3
 566post-handshake authentication. We should make sure to live up to that.
 567
 568See [curl issue 5396](https://github.com/curl/curl/issues/5396)
 569
 570## Support the `clienthello` extension
 571
 572Certain stupid networks and middle boxes have a problem with SSL handshake
 573packets that are within a certain size range because how that sets some bits
 574that previously (in older TLS version) were not set. The `clienthello`
 575extension adds padding to avoid that size range.
 576
 577- https://datatracker.ietf.org/doc/html/rfc7685
 578- [curl issue 2299](https://github.com/curl/curl/issues/2299)
 579
 580## Share the CA cache
 581
 582For TLS backends that supports CA caching, it makes sense to allow the share
 583object to be used to store the CA cache as well via the share API. Would allow
 584multiple easy handles to reuse the CA cache and save themselves from a lot of
 585extra processing overhead.
 586
 587## Add missing features to TLS backends
 588
 589The feature matrix at https://curl.se/libcurl/c/tls-options.html shows which
 590features are supported by which TLS backends, and thus also where there are
 591feature gaps.
 592
 593# Proxy
 594
 595## Retry SOCKS handshake on address type not supported
 596
 597When curl resolves a hostname, it might get a mix of IPv6 and IPv4 returned.
 598curl might then use an IPv6 address with a SOCKS5 proxy, which - if it does
 599not support IPv6 - returns "Address type not supported" and curl exits with
 600that error.
 601
 602Perhaps it is preferred if curl would in this situation instead first retry
 603the SOCKS handshake again for this case and then use one of the IPv4 addresses
 604for the target host.
 605
 606See [curl issue 17222](https://github.com/curl/curl/issues/17222)
 607
 608# Schannel
 609
 610## Extend support for client certificate authentication
 611
 612The existing support for the `-E`/`--cert` and `--key` options could be
 613extended by supplying a custom certificate and key in PEM format, see:
 614[Getting a Certificate for
 615Schannel](https://learn.microsoft.com/windows/win32/secauthn/getting-a-certificate-for-schannel)
 616
 617## Extend support for the `--ciphers` option
 618
 619The existing support for the `--ciphers` option could be extended by mapping
 620the OpenSSL/GnuTLS cipher suites to the Schannel APIs, see [Specifying
 621Schannel Ciphers and Cipher
 622Strengths](https://learn.microsoft.com/windows/win32/secauthn/specifying-schannel-ciphers-and-cipher-strengths).
 623
 624## Add option to allow abrupt server closure
 625
 626libcurl with Schannel errors without a known termination point from the server
 627(such as length of transfer, or SSL "close notify" alert) to prevent against a
 628truncation attack. Really old servers may neglect to send any termination
 629point. An option could be added to ignore such abrupt closures.
 630
 631[curl issue 4427](https://github.com/curl/curl/issues/4427)
 632
 633# SASL
 634
 635## Other authentication mechanisms
 636
 637Add support for other authentication mechanisms such as `OLP`, `GSS-SPNEGO`
 638and others.
 639
 640## Add `QOP` support to GSSAPI authentication
 641
 642Currently the GSSAPI authentication only supports the default `QOP` of auth
 643(Authentication), whilst Kerberos V5 supports both `auth-int` (Authentication
 644with integrity protection) and `auth-conf` (Authentication with integrity and
 645privacy protection).
 646
 647# SSH protocols
 648
 649## Multiplexing
 650
 651SSH is a perfectly fine multiplexed protocols which would allow libcurl to do
 652multiple parallel transfers from the same host using the same connection, much
 653in the same spirit as HTTP/2 does. libcurl however does not take advantage of
 654that ability but does instead always create a new connection for new transfers
 655even if an existing connection already exists to the host.
 656
 657To fix this, libcurl would have to detect an existing connection and "attach"
 658the new transfer to the existing one.
 659
 660## Handle growing SFTP files
 661
 662The SFTP code in libcurl checks the file size *before* a transfer starts and
 663then proceeds to transfer exactly that amount of data. If the remote file
 664grows while the transfer is in progress libcurl does not notice and does not
 665adapt. The OpenSSH SFTP command line tool does and libcurl could also attempt
 666to download more to see if there is more to get...
 667
 668[curl issue 4344](https://github.com/curl/curl/issues/4344)
 669
 670## Read keys from `~/.ssh/id_ecdsa`, `id_ed25519`
 671
 672The libssh2 backend in curl is limited to only reading keys from `id_rsa` and
 673`id_dsa`, which makes it fail connecting to servers that use more modern key
 674types.
 675
 676[curl issue 8586](https://github.com/curl/curl/issues/8586)
 677
 678## Support `CURLOPT_PREQUOTE`
 679
 680The two other `QUOTE` options are supported for SFTP, but this was left out
 681for unknown reasons.
 682
 683## SSH over HTTPS proxy for libssh
 684
 685The SSH based protocols SFTP and SCP did not work over HTTPS proxy at all
 686until [curl pull request 6021](https://github.com/curl/curl/pull/6021) brought
 687the functionality with the libssh2 backend. Presumably, this support can/could
 688be added for the libssh backend as well.
 689
 690## SFTP with `SCP://`
 691
 692OpenSSH 9 switched their `scp` tool to speak SFTP under the hood. Going
 693forward it might be worth having curl or libcurl attempt SFTP if SCP fails to
 694follow suite.
 695
 696# Command line tool
 697
 698## multi-threading
 699
 700When asked to do transfers in parallel, the curl tool could be extended to use
 701a number of independent worker threads. This would allow faster transfers in
 702situations where curl becomes CPU bound.
 703
 704Ideally, curl would (with permission) fire up new threads on demand when it
 705deems that it might be helpful. Perhaps, if it has more transfers to add and
 706the existing transfers make the CPU busy enough and there are more cores
 707available.
 708
 709## sync
 710
 711`curl --sync http://example.com/feed[1-100].rss` or
 712`curl --sync http://example.net/{index,calendar,history}.html`
 713
 714Downloads a range or set of URLs using the remote name, but only if the remote
 715file is newer than the local file. A `Last-Modified` HTTP date header should
 716also be used to set the mod date on the downloaded file.
 717
 718## glob posts
 719
 720Globbing support for `-d` and `-F`, as in `curl -d "name=foo[0-9]" URL`. This
 721is easily scripted though.
 722
 723## `--proxycommand`
 724
 725Allow the user to make curl run a command and use its stdio to make requests
 726and not do any network connection by itself. Example:
 727
 728    curl --proxycommand 'ssh pi@raspberrypi.local -W 10.1.1.75 80' \
 729      http://some/otherwise/unavailable/service.php
 730
 731See [curl issue 4941](https://github.com/curl/curl/issues/4941)
 732
 733## UTF-8 filenames in Content-Disposition
 734
 735RFC 6266 documents how UTF-8 names can be passed to a client in the
 736`Content-Disposition` header, and curl does not support this.
 737
 738[curl issue 1888](https://github.com/curl/curl/issues/1888)
 739
 740## Option to make `-Z` merge lined based outputs on stdout
 741
 742When a user requests multiple lined based files using `-Z` and sends them to
 743stdout, curl does not *merge* and send complete lines fine but may send
 744partial lines from several sources.
 745
 746[curl issue 5175](https://github.com/curl/curl/issues/5175)
 747
 748## specify which response codes that make `-f`/`--fail` return error
 749
 750Allows a user to better specify exactly which error code(s) that are fine and
 751which are errors for their specific uses cases
 752
 753## Choose the name of file in braces for complex URLs
 754
 755When using braces to download a list of URLs and you use complicated names
 756in the list of alternatives, it could be handy to allow curl to use other
 757names when saving.
 758
 759Consider a way to offer that. Possibly like
 760`{partURL1:name1,partURL2:name2,partURL3:name3}` where the name following the
 761colon is the output name.
 762
 763See [curl issue 221](https://github.com/curl/curl/issues/221)
 764
 765## improve how curl works in a Windows console window
 766
 767If you pull the scroll bar when transferring with curl in a Windows console
 768window, the transfer is interrupted and can get disconnected. This can
 769probably be improved. See [curl issue 322](https://github.com/curl/curl/issues/322)
 770
 771## Windows: set attribute 'archive' for completed downloads
 772
 773The archive bit (`FILE_ATTRIBUTE_ARCHIVE, 0x20`) separates files that shall be
 774backed up from those that are either not ready or have not changed.
 775
 776Downloads in progress are neither ready to be backed up, nor should they be
 777opened by a different process. Only after a download has been completed it is
 778sensible to include it in any integer snapshot or backup of the system.
 779
 780See [curl issue 3354](https://github.com/curl/curl/issues/3354)
 781
 782## keep running, read instructions from pipe/socket
 783
 784Provide an option that makes curl not exit after the last URL (or even work
 785without a given URL), and then make it read instructions passed on a pipe or
 786over a socket to make further instructions so that a second subsequent curl
 787invoke can talk to the still running instance and ask for transfers to get
 788done, and thus maintain its connection pool, DNS cache and more.
 789
 790## Acknowledge `Ratelimit` headers
 791
 792Consider a command line option that can make curl do multiple serial requests
 793while acknowledging server specified [rate
 794limits](https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-ratelimit-headers/).
 795
 796See [curl issue 5406](https://github.com/curl/curl/issues/5406)
 797
 798## `--dry-run`
 799
 800A command line option that makes curl show exactly what it would do and send
 801if it would run for real.
 802
 803See [curl issue 5426](https://github.com/curl/curl/issues/5426)
 804
 805## `--retry` should resume
 806
 807When `--retry` is used and curl actually retries transfer, it should use the
 808already transferred data and do a resumed transfer for the rest (when
 809possible) so that it does not have to transfer the same data again that was
 810already transferred before the retry.
 811
 812See [curl issue 1084](https://github.com/curl/curl/issues/1084)
 813
 814## retry on network is unreachable
 815
 816The `--retry` option retries transfers on *transient failures*. We later added
 817`--retry-connrefused` to also retry for *connection refused* errors.
 818
 819Suggestions have been brought to also allow retry on *network is unreachable*
 820errors and while totally reasonable, maybe we should consider a way to make
 821this more configurable than to add a new option for every new error people
 822want to retry for?
 823
 824[curl issue 1603](https://github.com/curl/curl/issues/1603)
 825
 826## hostname sections in config files
 827
 828config files would be more powerful if they could set different configurations
 829depending on used URLs, hostname or possibly origin. Then a default `.curlrc`
 830could a specific user-agent only when doing requests against a certain site.
 831
 832## retry on the redirected-to URL
 833
 834When curl is told to `--retry` a failed transfer and follows redirects, it
 835might get an HTTP 429 response from the redirected-to URL and not the original
 836one, which then could make curl decide to rather retry the transfer on that
 837URL only instead of the original operation to the original URL.
 838
 839Perhaps extra emphasized if the original transfer is a large POST that
 840redirects to a separate GET, and that GET is what gets the 529
 841
 842See [curl issue 5462](https://github.com/curl/curl/issues/5462)
 843
 844## Set the modification date on an uploaded file
 845
 846For SFTP and possibly FTP, curl could offer an option to set the modification
 847time for the uploaded file.
 848
 849See [curl issue 5768](https://github.com/curl/curl/issues/5768)
 850
 851## Use multiple parallel transfers for a single download
 852
 853To enhance transfer speed, downloading a single URL can be split up into
 854multiple separate range downloads that get combined into a single final
 855result.
 856
 857An ideal implementation would not use a specified number of parallel
 858transfers, but curl could:
 859- First start getting the full file as transfer A
 860- If after N seconds have passed and the transfer is expected to continue for
 861  M seconds or more, add a new transfer (B) that asks for the second half of
 862  A's content (and stop A at the middle).
 863- If splitting up the work improves the transfer rate, it could then be done
 864  again. Then again, etc up to a limit.
 865
 866This way, if transfer B fails (because Range: is not supported) it lets
 867transfer A remain the single one. N and M could be set to some sensible
 868defaults.
 869
 870See [curl issue 5774](https://github.com/curl/curl/issues/5774)
 871
 872## Prevent terminal injection when writing to terminal
 873
 874curl could offer an option to make escape sequence either non-functional or
 875avoid cursor moves or similar to reduce the risk of a user getting tricked by
 876clever tricks.
 877
 878See [curl issue 6150](https://github.com/curl/curl/issues/6150)
 879
 880## `-J` and `-O` with %-encoded filenames
 881
 882`-J`/`--remote-header-name` does not decode %-encoded filenames. RFC 6266
 883details how it should be done. The can of worm is that we have no charset
 884handling in curl and ASCII >=128 is a challenge for us. Not to mention that
 885decoding also means that we need to check for nastiness that is attempted,
 886like `../` sequences and the like. Probably everything to the left of any
 887embedded slashes should be cut off. See https://curl.se/bug/view.cgi?id=1294
 888
 889`-O` also does not decode %-encoded names, and while it has even less
 890information about the charset involved the process is similar to the `-J`
 891case.
 892
 893Note that we do not decode `-O` without the user asking for it with some other
 894means, since `-O` has always been documented to use the name exactly as
 895specified in the URL.
 896
 897## `-J` with `-C -`
 898
 899When using `-J` (with `-O`), automatically resumed downloading together with
 900`-C -` fails. Without `-J` the same command line works. This happens because
 901the resume logic is worked out before the target filename (and thus its
 902pre-transfer size) has been figured out. This can be improved.
 903
 904https://curl.se/bug/view.cgi?id=1169
 905
 906## `--retry` and transfer timeouts
 907
 908If using `--retry` and the transfer timeouts (possibly due to using -m or
 909`-y`/`-Y`) the next attempt does not resume the transfer properly from what
 910was downloaded in the previous attempt but truncates and restarts at the
 911original position where it was at before the previous failed attempt. See
 912https://curl.se/mail/lib-2008-01/0080.html
 913
 914# Build
 915
 916## Enable `PIE` and `RELRO` by default
 917
 918Especially when having programs that execute curl via the command line, `PIE`
 919renders the exploitation of memory corruption vulnerabilities a lot more
 920difficult. This can be attributed to the additional information leaks being
 921required to conduct a successful attack. `RELRO`, on the other hand, masks
 922different binary sections like the `GOT` as read-only and thus kills a handful
 923of techniques that come in handy when attackers are able to arbitrarily
 924overwrite memory. A few tests showed that enabling these features had close to
 925no impact, neither on the performance nor on the general functionality of
 926curl.
 927
 928## Do not use GNU libtool on OpenBSD
 929
 930When compiling curl on OpenBSD with `--enable-debug` it gives linking errors
 931when you use GNU libtool. This can be fixed by using the libtool provided by
 932OpenBSD itself. However for this the user always needs to invoke make with
 933`LIBTOOL=/usr/bin/libtool`. It would be nice if the script could have some
 934logic to detect if this system is an OpenBSD host and then use the OpenBSD
 935libtool instead.
 936
 937See [curl issue 5862](https://github.com/curl/curl/issues/5862)
 938
 939## Package curl for Windows in a signed installer
 940
 941See [curl issue 5424](https://github.com/curl/curl/issues/5424)
 942
 943## make configure use `--cache-file` more and better
 944
 945The configure script can be improved to cache more values so that repeated
 946invokes run much faster.
 947
 948See [curl issue 7753](https://github.com/curl/curl/issues/7753)
 949
 950# Test suite
 951
 952## SSL tunnel
 953
 954Make our own version of stunnel for simple port forwarding to enable HTTPS and
 955FTP-SSL tests without the stunnel dependency, and it could allow us to provide
 956test tools built with either OpenSSL or GnuTLS
 957
 958## more protocols supported
 959
 960Extend the test suite to include more protocols. The telnet could do FTP or
 961http operations (for which we have test servers).
 962
 963## more platforms supported
 964
 965Make the test suite work on more platforms. OpenBSD and macOS. Remove fork()s
 966and it should become even more portable.
 967
 968## write an SMB test server to replace impacket
 969
 970This would allow us to run SMB tests on more platforms and do better and more
 971covering tests.
 972
 973See [curl issue 15697](https://github.com/curl/curl/issues/15697)
 974
 975## Use the RFC 6265 test suite
 976
 977A test suite made for HTTP cookies (RFC 6265) by Adam Barth [is
 978available](https://github.com/abarth/http-state/tree/master/tests).
 979
 980It would be good if someone would write a script/setup that would run curl
 981with that test suite and detect deviance. Ideally, that would even be
 982incorporated into our regular test suite.
 983
 984## Run web-platform-tests URL tests
 985
 986Run web-platform-tests URL tests and compare results with browsers on
 987`wpt.fyi`.
 988
 989It would help us find issues to fix and help us document where our parser
 990differs from the WHATWG URL spec parsers.
 991
 992See [curl issue 4477](https://github.com/curl/curl/issues/4477)
 993
 994# MQTT
 995
 996## Support rate-limiting
 997
 998The rate-limiting logic is done in the PERFORMING state in multi.c but MQTT is
 999not (yet) implemented to use that.
1000
1001## Support MQTTS
1002
1003## Handle network blocks
1004
1005Running test suite with `CURL_DBG_SOCK_WBLOCK=90 ./runtests.pl -a mqtt` makes
1006several MQTT test cases fail where they should not.
1007
1008## large payloads
1009
1010libcurl unnecessarily allocates heap memory to hold the entire payload to get
1011sent, when the data is already perfectly accessible where it is when
1012`CURLOPT_POSTFIELDS` is used. This is highly inefficient for larger payloads.
1013Additionally, libcurl does not support using the read callback for sending
1014MQTT which is yet another way to avoid having to hold large payload in memory.
1015
1016# TFTP
1017
1018## TFTP does not convert LF to CRLF for `mode=netascii`
1019
1020RFC 3617 defines that an TFTP transfer can be done using `netascii` mode. curl
1021does not support extracting that mode from the URL nor does it treat such
1022transfers specifically. It should probably do LF to CRLF translations for
1023them.
1024
1025See [curl issue 12655](https://github.com/curl/curl/issues/12655)
1026
1027# Gopher
1028
1029## Handle network blocks
1030
1031Running test suite with `CURL_DBG_SOCK_WBLOCK=90 ./runtests.pl -a 1200 to
10321300` makes several Gopher test cases fail where they should not.