lights FAQ Forum github.com/luapower/libcurl
This package
libcurl

Networking
luasec
resolver
libcurl
rsync

libcurl

URL transfers


local curl = require'libcurl'

libcurl is a client-side URL transfer library, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos), file transfer resume, http proxy tunneling and more!

API

easy interface
curl.easy(url | {opt=val}) -> etr create an easy transfer
etr:set(opt,val | {opt=val}) -> etr set option(s)
etr:perform() -> etr | nil,err,ecode perform the transfer
etr:close() close the transfer
etr:clone([url | {opt=val}]) -> etr clone a transfer
etr:reset([{opt=val}]) -> etr reset all options to their default values
etr:info(opt) -> val get info about the transfer
etr:recv(buf, bufsize) -> n | nil,err,errcode receive raw data
etr:send(buf, bufsize) -> n | nil,err,errcode send raw data
etr:escape(s) -> s|nil escape URL
etr:unescape(s) -> s|nil unescape URL
etr:pause([flags]) pause transfer
multi interface
curl.multi([{opt=val}]) -> mtr create a multi transfer
mtr:set(opt,val | {opt=val}) -> mtr set option(s)
mtr:add(etr) -> mtr add an easy transfer to the queue
mtr:remove(etr) -> mtr remove an easy transfer to the queue
mtr:perform() -> transfers_left | nil,err,ecode start/keep transfering
mtr:close() close the transfer
mtr:wait([timeout_seconds], [extra_fds, extra_nfds]) -> numfds | nil,err,errcode poll on all handles
mtr:fdset(read_fd_set, write_fd_set, exc_fd_set) -> max_fd | nil,err,errcode get file descriptors
mtr:timeout() -> seconds | nil how long to wait for socket actions
mtr:info_read() -> CURLMsg*|nil, msgs_in_queue read multi stack info
mtr:socket_action() read/write available data given an action
mtr:assign(sockfd, p) -> mtr set data to associate with an internal socket
share interface
curl.share([{opt=val}]) -> shr create a share object
shr:set(opt,val | {opt=val}) -> shr set option(s)
shr:free() free the share object
multipart forms
etr:mime() -> mime create a mime object
mime:free() free a mime object
mime:part() -> mimepart add a mime part to a mime object
mimepart:name(name) set mime part’s name
mimepart:filename(filename) set mime part’s filename
mimepart:type(mimetype) set mime part’s content type
mimepart:encoder(encoder) set mime part’s transfer encoding
mimepart:data(s[, sz]) get mime part’s data from string or cdata buffer
mimepart:file(file) get mime part’s data from file
mimepart:data_cb(sz, read, seek, free, arg) get mime part’s data with callbacks
mimepart:subparts({mime1,...}) set mime part’s subparts
mimepart:headers({header=value}) set mime part’s headers
misc.
curl.C the libcurl ffi clib object/namespace
curl.init([opt]) global init
curl.init{malloc=,free=,realloc=...} global init with custom allocators
curl.free() global cleanup
curl.version() -> s get version info as a string
curl.version_info([ver]) -> t get detailed version info as a table
curl.checkver(maj[, min[, patch]]) -> true|false check if CURL version is >= maj.min.patch
curl.getdate(s) -> timestamp parse a date/time to a Unix timestamp
curl.easy.strerror(errcode) -> errmsg look-up an easy interface error code
curl.multi.strerror(errcode) -> errmsg look-up a multi interface error code
curl.share.strerror(errcode) -> errmsg look-up a share interface error code
curl.type(x) -> 'easy'|'multi'|'share'|nil get curl object type

Easy vs multi interface

The easy interface is synchronous while the multi interface can do multiple transfers asynchronously. A multi transfer is set up as a list of easy transfers.

Easy interface

curl.easy(url | {opt=val}) -> etr

Create a transfer using the easy interface. Options are below (they also go for etr:set()). All options are assumed immutable and the option values are anchored internally for the lifetime of the transfer object.

How option values are converted:

  • long options can be given as Lua numbers.
  • char* options can be given as Lua strings (anchored).
  • off_t options can be given as Lua numbers (no need for the _long suffix).
  • Enum options can be given as strings (case-insensitive, no prefix).
  • Bitmask options can be given as tables of form {mask_name = true|false} (again, the mask name follows the C name but case-insensitive and without the prefix).
  • curl_slist options can be given as lists of strings (anchored).
  • Callbacks can be given as Lua functions (anchored). The ffi callback objects are freed on etr:free() (*).

(*) Callback objects are ref-counted which means that replacing them on cloned transfers does not result in double-frees, and freeing them is deterministic, which is important since their number is hard-limited.

Main options
url URL to work on.
curlu set URL with CURLU object
protocols Allowed protocols.
redir_protocols Protocols to allow redirects to.
default_protocol Default protocol.
port Port number to connect to.
range Request range: “X-Y,N-M,…”.
resume_from Resume transfer from offset.
header Include the header in the body output.
maxfilesize Max. file size to get.
upload Enable data upload.
infilesize Size of file to send.
timecondition Make a time-conditional request.
timevalue Timestamp for conditional request.
Progress Tracking
noprogress Shut off the progress meter.
progressfunction Callback for progress meter.
progressdata Data pointer to pass to the progress meter callback.
xferinfofunction Callback for progress meter.
xferinfodata Data pointer to pass to the progress meter callback.
Error Handling
verbose Display verbose information.
stderr stderr replacement stream.
errorbuffer Error message buffer.
failonerror Fail on HTTP 4xx errors.
Proxies
proxy Proxy to use.
proxyport Proxy port to use.
proxytype Proxy type.
noproxy Filter out hosts from proxy use.
httpproxytunnel Tunnel through the HTTP proxy.
socks5_gssapi_service Socks5 GSSAPI service name.
socks5_gssapi_nec Socks5 GSSAPI NEC mode.
socks5_auth set allowed methods for SOCKS5 proxy authentication
proxy_service_name Proxy service name.
proxy_cainfo path to proxy Certificate Authority (CA) bundle
proxy_capath directory holding proxy CA certificates
proxy_ssl_verifypeer verify the proxy’s SSL certificate
proxy_ssl_verifyhost verify the proxy certificate’s name against host
proxy_sslversion set preferred proxy TLS/SSL version
proxy_tlsauth_username user name to use for proxy TLS authentication
proxy_tlsauth_password password to use for proxy TLS authentication
proxy_tlsauth_type set proxy TLS authentication methods
proxy_sslcert set SSL proxy client certificate
proxy_sslcerttype specify type of the proxy client SSL certificate
proxy_sslkey specify private keyfile for TLS and SSL proxy client cert
proxy_sslkeytype set type of the proxy private key file
proxy_keypasswd set passphrase to proxy private key
proxy_ssl_cipher_list specify ciphers to use for proxy TLS
proxy_crlfile specify a proxy Certificate Revocation List file
proxy_ssl_options set proxy SSL behavior options
pre_proxy set pre-proxy to use
proxy_pinnedpublickey set pinned public key for https proxy
suppress_connect_headers Suppress proxy CONNECT response headers from user callbacks
haproxyprotocol send HAProxy PROXY protocol v1 header
proxy_tls13_ciphers Ciphers suites for proxy TLS 1.3
I/O Callbacks
writefunction Callback for writing data.
writedata Data pointer to pass to the write callback.
readfunction Callback for reading data.
readdata Data pointer to pass to the read callback.
seekfunction Callback for seek operations.
seekdata Data pointer to pass to the seek callback.
Speed Limits
low_speed_limit Low speed limit to abort transfer.
low_speed_time Time to be below the speed to trigger low speed abort.
max_send_speed Cap the upload speed.
max_recv_speed Cap the download speed.
Authentication
netrc Enable .netrc parsing.
netrc_file .netrc file name.
userpwd User name and password.
proxyuserpwd Proxy user name and password.
username User name.
password Password.
login_options Login options.
proxyusername Proxy user name.
proxypassword Proxy password.
httpauth HTTP server authentication methods.
tlsauth_username TLS authentication user name.
tlsauth_password TLS authentication password.
tlsauth_type TLS authentication methods.
proxyauth HTTP proxy authentication methods.
sasl_ir Enable SASL initial response.
xoauth2_bearer OAuth2 bearer token.
sasl_authzid Authorisation identity (identity to act as)
HTTP Protocol
autoreferer Automatically set Referer: header.
accept_encoding Accept-Encoding and automatic decompressing data.
transfer_encoding Request Transfer-Encoding.
followlocation Follow HTTP redirects.
unrestricted_auth Do not restrict authentication to original host.
maxredirs Maximum number of redirects to follow.
postredir How to act on redirects after POST.
put Issue a HTTP PUT request.
post Issue a HTTP POST request.
httpget Do a HTTP GET request.
postfields Send a POST with this data.
postfieldsize The POST data is this big.
copypostfields Send a POST with this data - and copy it.
httppost Multipart formpost HTTP POST.
referer Referer: header.
useragent User-Agent: header.
httpheader Custom HTTP headers.
headeropt Control custom headers.
proxyheader Custom HTTP headers sent to proxy.
http200aliases Alternative versions of 200 OK.
cookie Cookie(s) to send.
cookiefile File to read cookies from.
cookiejar File to write cookies to.
cookiesession Start a new cookie session.
cookielist Add or control cookies.
http_version HTTP version to use.
ignore_content_length Ignore Content-Length.
http_content_decoding Disable Content decoding.
http_transfer_decoding Disable Transfer decoding.
expect_100_timeout_ms 100-continue timeout in ms.
pipewait Wait on connection to pipeline on it.
keep_sending_on_error Keep sending on early HTTP response >= 300
request_target Specify an alternative target for this request
mimepost Set post/send data from mime structure
disallow_username_in_url Disallow specifying username in the url
trailerfunction Set callback for sending trailing headers
trailerdata Custom pointer passed to the trailing headers callback
http09_allowed Allow HTTP/0.9 response
HTTP/2 Protocol
stream_weight Set numerical stream weight
stream_depends Set stream this transfer depends on
stream_depends_e Set stream this transfer depends on exclusively
Connection
interface Bind connection locally to this.
localport Bind connection locally to this port.
localportrange Bind connection locally to port range.
tcp_nodelay Disable the Nagle algorithm.
tcp_keepalive Enable TCP keep-alive.
tcp_keepidle Idle time before sending keep-alive.
tcp_keepintvl Interval between keep-alive probes.
tcp_fastopen enable TCP Fast Open
address_scope IPv6 scope for local addresses.
unix_socket_path Path to a Unix domain socket.
abstract_unix_socket Set an abstract Unix domain socket
dns_interface Bind name resolves to an interface.
dns_cache_timeout Timeout for DNS cache.
dns_local_ip4 Bind name resolves to an IP4 address.
dns_local_ip6 Bind name resolves to an IP6 address.
dns_servers Preferred DNS servers.
dns_use_global_cache OBSOLETE Enable global DNS cache.
timeout Timeout for the entire request in seconds.
timeout_ms Timeout for the entire request in ms.
connecttimeout Timeout for the connection phase in seconds.
connecttimeout_ms Timeout for the connection phase in ms.
accepttimeout_ms Timeout for a connection be accepted.
server_response_timeout
fresh_connect Use a new connection.
forbid_reuse Prevent subsequent connections from re-using this connection.
connect_only Only connect, nothing else.
resolve Provide fixed/fake name resolves.
conv_from_network_function Callback for code base conversion.
conv_to_network_function Callback for code base conversion.
conv_from_utf8_function Callback for code base conversion.
opensocketfunction
opensocketdata
closesocketfunction
closesocketdata
sockoptfunction Callback for sockopt operations.
sockoptdata Data pointer to pass to the sockopt callback.
connect_to Connect to a specific host and port instead of the URL’s host and port
happy_eyeballs_timeout_ms Head start for ipv6 for happy eyeballs
resolver_start_function Set callback to be called before a new resolve request is started
resolver_start_data Custom pointer passed to the resolver start callback
dns_shuffle_addresses Shuffle addresses when a hostname returns more than one
doh_url Provide the DNS-over-HTTPS URL
upkeep_interval_ms Connection upkeep interval
maxage_conn Max idle time allowed for reusing a connection
SSH Protocol
ssh_auth_types SSH authentication types.
ssh_public_keyfile File name of public key.
ssh_private_keyfile File name of private key.
ssh_knownhosts File name with known hosts.
ssh_keyfunction Callback for known hosts handling.
ssh_keydata Custom pointer to pass to ssh key callback.
ssh_host_public_key_md5 MD5 of host’s public key.
ssh_compression enables compression / decompression of SSH traffic
SMTP Protocol
mail_from Address of the sender.
mail_rcpt Address of the recipients.
mail_auth Authentication address.
TFTP Protocol
tftp_blksize TFTP block size.
tftp_no_options Do not send TFTP options requests.
SSL
use_ssl Use TLS/SSL.
sslcert Client cert.
sslversion SSL version to use.
sslcerttype Client cert type.
sslkey Client key.
sslkeytype Client key type.
keypasswd Client key password.
sslengine Use identifier with SSL engine.
sslengine_default Default SSL engine.
ssl_options Control SSL behavior.
ssl_falsestart Enable TLS False Start.
ssl_cipher_list Ciphers to use.
ssl_verifyhost Verify the host name in the SSL certificate.
ssl_verifypeer Verify the SSL certificate.
ssl_verifystatus Verify the SSL certificate’s status.
ssl_ctx_function Callback for SSL context logic.
ssl_ctx_data Data pointer to pass to the SSL context callback.
ssl_sessionid_cache Disable SSL session-id cache.
ssl_enable_npn
ssl_enable_alpn
cainfo CA cert bundle.
capath Path to CA cert bundle.
crlfile Certificate Revocation List.
issuercert Issuer certificate.
certinfo Extract certificate info.
pinnedpublickey Set pinned SSL public key.
krblevel Kerberos security level.
random_file Provide source for entropy random data.
egdsocket Identify EGD socket for entropy.
gssapi_delegation Disable GSS-API delegation.
tls13_ciphers Specify ciphers suites to use for TLS 1.3
altsvc_ctrl Control alt-svc behavior
altsvc Set alt-svc cache file name
FTP Protocol
ftpport Use active FTP.
quote Commands to run before transfer.
postquote Commands to run after transfer.
prequote Commands to run just before transfer.
append Append to remote file.
ftp_use_eprt Use EPTR.
ftp_use_epsv Use EPSV.
ftp_use_pret Use PRET.
ftp_create_missing_dirs Create missing directories on the remote server.
ftp_response_timeout Timeout for FTP responses.
ftp_alternative_to_user Alternative to USER.
ftp_skip_pasv_ip Ignore the IP address in the PASV response.
ftpsslauth Control how to do TLS.
ftp_ssl_ccc Back to non-TLS again after authentication.
ftp_account Send ACCT command.
ftp_filemethod Specify how to reach files.
transfertext Use text transfer.
proxy_transfer_mode Add transfer mode to URL over proxy.
RTSP Protocol
rtsp_request RTSP request.
rtsp_session_id RTSP session-id.
rtsp_stream_uri RTSP stream URI.
rtsp_transport RTSP Transport:
rtsp_client_cseq Client CSEQ number.
rtsp_server_cseq CSEQ number for RTSP Server->Client request.
interleavefunction Callback for RTSP interleaved data.
interleavedata Data pointer to pass to the RTSP interleave callback.
Misc. Options
path_as_is Disable squashing /../ and /./ sequences in the path.
buffersize Ask for smaller buffer size.
nosignal Do not install signal handlers.
share Share object to use.
private Private pointer to store.
ipresolve IP version to resolve to.
ioctlfunction Callback for I/O operations.
ioctldata Data pointer to pass to the I/O callback.
service_name SPNEGO service name.
crlf Convert newlines.
customrequest Custom request/method.
filetime Request file modification date and time.
dirlistonly List only.
nobody Do not get the body contents.
new_file_perms Mode for creating new remote files.
new_directory_perms Mode for creating new remote directories.
chunk_bgn_function Callback for wildcard download start of chunk.
chunk_end_function Callback for wildcard download end of chunk.
chunk_data Data pointer to pass to the chunk callbacks.
fnmatch_function Callback for wildcard matching.
fnmatch_data Data pointer to pass to the wildcard matching callback.
wildcardmatch Transfer multiple files according to a file name pattern.
telnetoptions TELNET options.
maxconnects Maximum number of connections in the connection pool.
headerfunction Callback for writing received headers.
headerdata Data pointer to pass to the header callback.
upload_buffersize Set preferred upload buffer size
Debugging
debugfunction Callback for debug information.
debugdata Data pointer to pass to the debug callback.

Multi interface

curl.multi([{etr1,..., opt=val}]) -> mtr

Create a transfer using the multi interface. The details are the same as with the easy interface except that the list of options is different (see below). Also, if the options table has any elements in the array part, mtr:add() is called for/with each element.

There’s a section on the libcurl tutorial on how to use the multi interface.

socketfunction Callback about what to wait for
socketdata Custom pointer passed to the socket callback
pipelining enable/disable HTTP pipelining
timerfunction Set callback to receive timeout values
timerdata Custom pointer to pass to timer callback
maxconnects Set size of connection cache
max_host_connections Set max number of connections to a single host
max_pipeline_length Maximum number of requests in a pipeline
content_length_penalty_size Size threshold for pipelining penalty
chunk_length_penalty_size Chunk length threshold for pipelining
pipelining_site_bl Pipelining host blacklist (list of strings)
pipelining_server_bl Pipelining server blacklist (list of strings)
max_total_connections Max simultaneously open connections
pushfunction Callback that approves or denies server pushes
pushdata Pointer to pass to push callback

Share interface

curl.share([{opt=val}]) -> shr

Create a share object. Options below (also for shr:set()):

lockfunc lock data callback
unlockfunc unlock data callback
share what tyoe of data to share
unshare what type of data not to share
userdata pointer to pass to lock and unlock functions

Binaries

The included libcurl binaries are compiled to use the SSL/TLS APIs provided by the host OS by default but can also use luapower’s openssl as SSL backend provided libcurl is initalized manually with curl.init{sslbackend='openssl'}.

The decision to ship OpenSSL even on Linux is because OpenSSL is not ABI-compatible between versions and distros don’t usually ship multiple versions of it (sigh, Linux).

DNS resolving is asynchronous using the multi-threaded resolver.

Features that were not compiled in:

  • SFTP and SCP (requires libssh2).
  • HTTP2 (requires ng-http2).
  • IDN (requires libidn).
  • RTMP (requires librtmp).
  • PSL (requires libpsl).
  • LDAP.

Last updated: 10 months ago | Edit on GitHub

Package:libcurl
Pkg type:Lua+ffi
Version: 729f76b
Last commit:
Author: Cosmin Apreutesei
License: MIT
Import ver: 7.67.0

Requires: luajit  openssl 

Required by: none


Top