changelog

Changelog
---------

 - CHANGE: REMOVED: LWS_WITH_GENRSA, LWS_WITH_GENHASH, LWS_WITH_GENEC,
 LWS_WITH_GENAES have all been removed and combined into LWS_WITH_GENCRYPTO

 - CHANGE: REMOVED: LWS_WITH_JWS, LWS_WITH_JWE have been removed and combined
 into LWS_WITH_JOSE

v3.1.0
======

 - CHANGE: REMOVED: lws_client_connect() and lws_client_connect_extended()
   compatibility apis for lws_client_connect_via_info() have been marked as
   deprecated for several versions and are now removed.  Use
   lws_client_connect_via_info() directly instead.

 - CHANGE: CMAKE:
     - LWS_WITH_HTTP2:         now defaults ON

 - CHANGE: Minimal examples updated to use Content Security Policy best
   practices, using
   `LWS_SERVER_OPTION_HTTP_HEADERS_SECURITY_BEST_PRACTICES_ENFORCE` vhost
   option flag and disabling of inline style and scripts.  A side-effect of
   this is that buffers used to marshal headers have to be prepared to take
   more content than previously... LWS_RECOMMENDED_MIN_HEADER_SPACE (2048
   currently) is available for user (and internal) use to logically tie the
   buffer size to this usecase (and follow future increases).

 - NEW: CMAKE
     - LWS_FOR_GITOHASHI: sets various cmake options suitable for gitohashi
     - LWS_WITH_ASAN: for Linux, enable build with ASAN

     Don't forget LWS_WITH_DISTRO_RECOMMENDED, which enables a wide range of lws
     options suitable for a distro build of the library.
     
 - NEW: lws threadpool - lightweight pool of pthreads integrated to lws wsi, with
   all synchronization to event loop handled internally, queue for excess tasks
   [threadpool docs](https://libwebsockets.org/git/libwebsockets/tree/lib/misc/threadpool)
   [threadpool minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/ws-server/minimal-ws-server-threadpool)
   Cmake config: `-DLWS_WITH_THREADPOOL=1`

 - NEW: libdbus support integrated on lws event loop
   [lws dbus docs](https://libwebsockets.org/git/libwebsockets/tree/lib/roles/dbus)
   [lws dbus client minimal examples](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/dbus-client)
   [lws dbus server minimal examples](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/dbus-server)
   Cmake config: `-DLWS_ROLE_DBUS=1`

 - NEW: lws allocated chunks (lwsac) - helpers for optimized mass allocation of small
   objects inside a few larger malloc chunks... if you need to allocate a lot of
   inter-related structs for a limited time, this removes per-struct allocation
   library overhead completely and removes the need for any destruction handling
   [lwsac docs](https://libwebsockets.org/git/libwebsockets/tree/lib/misc/lwsac)
   [lwsac minimal example](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-lwsac)
   Cmake Config: `-DLWS_WITH_LWSAC=1`

 - NEW: lws tokenizer - helper api for robustly tokenizing your own strings without
   allocating or adding complexity.  Configurable by flags for common delimiter
   sets and comma-separated-lists in the tokenizer.  Detects and reports syntax
   errors.
   [lws_tokenize docs](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-tokenize.h)
   [lws_tokenize minimal example / api test](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-lws_tokenize)

 - NEW: lws full-text search - optimized trie generation, serialization,
   autocomplete suggestion generation and instant global search support extensible
   to huge corpuses of UTF-8 text while remaining super lightweight on resources.
   [full-text search docs](https://libwebsockets.org/git/libwebsockets/tree/lib/misc/fts)
   [full-text search minimal example / api test](https://libwebsockets.org/git/libwebsockets/tree/minimal-examples/api-tests/api-test-fts)
   [demo](https://libwebsockets.org/ftsdemo/)
   [demo sources](https://libwebsockets.org/git/libwebsockets/tree/plugins/protocol_fulltext_demo.c)
   Cmake config: `-DLWS_WITH_FTS=1 -DLWS_WITH_LWSAC=1`

 - NEW: gzip + brotli http server-side compression - h1 and h2 automatic advertising
   of server compression and application to files with mimetypes "text/*",
   "application/javascript" and "image/svg.xml".
   Cmake config: `-DLWS_WITH_HTTP_STREAM_COMPRESSION=1`, `-DLWS_WITH_HTTP_BROTLI=1`

 - NEW: managed disk cache - API for managing a directory containing cached files
   with hashed names, and automatic deletion of LRU files once the cache is
   above a given limit.
   [lws diskcache docs](https://libwebsockets.org/git/libwebsockets/tree/include/libwebsockets/lws-diskcache.h)
   Cmake config: `-DLWS_WITH_DISKCACHE=1`

 - NEW: http reverse proxy - lws mounts support proxying h1 or h2 requests to
   a local or remote IP, or unix domain socket over h1.  This allows microservice
   type architectures where parts of the common URL space are actually handled
   by external processes which may be remote or on the same machine.
   [lws gitohashi serving](https://libwebsockets.org/git/) is handled this way.
   CMake config: `-DLWS_WITH_HTTP_PROXY=1`
   
 - NEW: lws_buflist - internally several types of ad-hoc malloc'd buffer have
   been replaced by a new, exported api `struct lws_buflist`.  This allows
   multiple buffers to be chained and drawn down in strict FIFO order.

 - NEW: In the case of h1 upgrade, the connection header is checked to contain
   "upgrade".   The vhost flag LWS_SERVER_OPTION_VHOST_UPG_STRICT_HOST_CHECK
   also causes the Host: header to be confirmed to match the vhost name and
   listen port.
   
 - NEW: If no 404 redirect for `lws_return_http_status()` is specified for the vhost,
   the status page produced will try to bring in a stylesheet `/error.css`.  This allows
   you to produce styled 404 or other error pages with logos, graphics etc.  See
   https://libwebsockets.org/git/badrepo for an example of what you can do with it.

v3.0.0
======

 - CHANGE: Clients used to call LWS_CALLBACK_CLOSED same as servers...
   LWS_CALLBACK_CLIENT_CLOSED has been introduced and is called for clients
   now.
   
 - CHANGE: LWS_CALLBACK_CLIENT_CONNECTION_ERROR used to only be directed at
   protocols[0].  However in many cases, the protocol to bind to was provided
   at client connection info time and the wsi bound accordingly.  In those
   cases, CONNECTION_ERROR is directed at the bound protocol, not protcols[0]
   any more.

 - CHANGE: CMAKE: the following cmake defaults have changed with this version:
 
     - LWS_WITH_ZIP_FOPS:      now defaults OFF
     - LWS_WITH_RANGES:        now defaults OFF
     - LWS_WITH_ZLIB:          now defaults OFF
     - LWS_WITHOUT_EXTENSIONS: now defaults ON
     
 - CHANGE: REMOVED: lws_alloc_vfs_file() (read a file to malloc buffer)
 
 - CHANGE: REMOVED: lws_read() (no longer useful outside of lws internals)
 
 - CHANGE: REMOVED: ESP8266... ESP32 is now within the same price range and much
   more performant
   
 - CHANGE: soname bump... don't forget to `ldconfig`
     
 - NEW: all event libraries support "foreign" loop integration where lws itself
   if just a temporary user of the loop unrelated to the actual loop lifecycle.
   
   See `minimal-http-server-eventlib-foreign` for example code demonstrating
   this for all the event libraries.
   
   Internal loop in lws is also supported and demonstrated by
   `minimal-http-server-eventlib`.
 
 - NEW: ws-over-h2 support.  This is a new RFC-on-the-way supported by Chrome
   and shortly firefox that allows ws connections to be multiplexed back to the
   server on the same tcp + tls wrapper h2 connection that the html and scripts
   came in on.  This is hugely faster that discrete connections.
 
 - NEW: UDP socket adoption and related event callbacks
 
 - NEW: Multi-client connection binding, queuing and pipelining support.
 
   Lws detects multiple client connections to the same server and port, and
   optimizes how it handles them according to the server type and provided
   flags.  For http/1.0, all occur with individual parallel connections.  For
   http/1.1, you can enable keepalive pipelining, so the connections occur
   sequentially on a single network connection.  For http/2, they all occur
   as parallel streams within a single h2 network connection.
   
   See minimal-http-client-multi for example code. 
   
 - NEW: High resolution timer API for wsi, get a callback on your wsi with
   LWS_CALLBACK_TIMER, set and reset the timer with lws_set_timer_usecs(wsi, us)
   Actual resolution depends on event backend.  Works with all backends, poll,
   libuv, libevent, and libev.
   
 - NEW: Protocols can arrange vhost-protocol instance specific callbacks with
   second resolution using `lws_timed_callback_vh_protocol()`

 - NEW: ACME client plugin for self-service TLS certificates
  
 - NEW: RFC7517 JSON Web Keys RFC7638 JWK thumbprint, and RFC7515 JSON Web
    signatures support
  
 - NEW: lws_cancel_service() now provides a generic way to synchronize events
   from other threads, which appear as a LWS_CALLBACK_EVENT_WAIT_CANCELLED
   callback on all protocols.  This is compatible with all the event libraries.

 - NEW: support BSD poll() where changes to the poll wait while waiting are
   undone.

 - NEW: Introduce generic hash, hmac and RSA apis that operate the same
   regardless of OpenSSL or mbedTLS tls backend
  
 - NEW: Introduce X509 element query api that works the same regardless of
   OpenSSL or mbedTLS tls backend
    
 - NEW: Introduce over 30 "minimal examples" in ./minimal-examples... these
   replace most of the old test servers
   
    - test-echo -> minimal-ws-server-echo and minimal-ws-client-echo

    - test-server-libuv / -libevent / -libev ->
         minimal-https-server-eventlib / -eventlib-foreign / -eventlib-demos

    - test-server-v2.0 -> folded into all the minimal servers

    - test-server direct http serving -> minimal-http-server-dynamic
    
   The minimal examples allow individual standalone build using their own
   small CMakeLists.txt.
   
 - NEW: lws now detects any back-to-back writes that did not go through the
   event loop inbetween and reports them.  This will flag any possibility of
   failure rather than wait until the problem happens.
   
 - NEW: CMake has LWS_WITH_DISTRO_RECOMMENDED to select features that are
   appropriate for distros
   
 - NEW: Optional vhost URL `error_document_404` if given causes a redirect there
   instead of serve the default 404 page.
   
 - NEW: lws_strncpy() wrapper guarantees NUL in copied string even if it was
   truncated to fit.
   
 - NEW: for client connections, local protocol binding name can be separated
   from the ws subprotocol name if needed, using .local_protocol_name

 - NEW: Automatic detection of time discontiguities
   
 - NEW: Applies TCP_USER_TIMEOUT for Linux tcp keepalive where available
     
 - QA: 1600 tests run on each commit in Travis CI, including almost all
   Autobahn in client and server mode, various h2load tests, h2spec, attack.sh
   the minimal example selftests and others.

 - QA: fix small warnings introduced on gcc8.x (eg, Fedora 28)
 
 - QA: Add most of -Wextra on gcc (-Wsign-compare, -Wignored-qualifiers,
   -Wtype-limits, -Wuninitialized)
   
 - QA: clean out warnings on windows
 
 - QA: pass all 146 h2spec tests now on strict
 
 - QA: introduce 35 selftests that operate different minimal examples against
   each other and confirm the results.
 
 - QA: LWS_WITH_MINIMAL_EXAMPLES allows mass build of all relevant minimal-
   examples with the LWS build, for CI and to make all the example binaries
   available from the lws build dir ./bin
 
 - REFACTOR: the lws source directory layout in ./lib has been radically
   improved, and there are now README.md files in selected subdirs with extra
   documentation of interest to people working on lws itself.

 - REFACTOR: pipelined transactions return to the event loop before starting the
   next part. 
 
 - REFACTOR: TLS: replace all TLS library constants with generic LWS ones and
   adapt all the TLS library code to translate to these common ones.
   
   Isolated all the tls-related private stuff in `./lib/tls/private.h`, and all
   the mbedTLS stuff in `./lib/tls/mbedtls` + openSSL stuff in
   `./lib/tls/openssl`
   
 - REFACTOR: the various kinds of wsi possible with lws have been extracted
   from the main code and isolated into "roles" in `./lib/roles` which
   communicate with the core code via an ops struct.  Everything related to
   ah is migrated to the http role.
   
   wsi modes are eliminated and replaced by the ops pointer for the role the
   wsi is performing.  Generic states for wsi are available to control the
   lifecycle using core code.
   
   Adding new "roles" is now much easier with the changes and ops struct to
   plug into.

 - REFACTOR: reduce four different kinds of buffer management in lws into a
   generic scatter-gather struct lws_buflist. 

 - REFACTOR: close notifications go through event loop


v2.4.0
======

 - HTTP/2 server support is now mature and usable!  LWS_WITH_HTTP2=1 enables it.
   Uses ALPN to serve HTTP/2, HTTP/1 and ws[s] connections all from the same
   listen port seamlessly.  (Requires ALPN-capable OpenSSL 1.1 or mbedTLS).

 - LWS_WITH_MBEDTLS=1 at CMake now builds and works against mbedTLS instead of
   OpenSSL.  Most things work identically, although on common targets where
   OpenSSL has acceleration, mbedTLS is many times slower in operation.  However
   it is a lot smaller codewise.
   
 - Generic hash apis introduced that work the same on mbedTLS or OpenSSL backend
 
 - LWS_WITH_PEER_LIMITS tracks IPs across all vhosts and allows restrictions on
   both the number of simultaneous connections and wsi in use for any single IP

 - lws_ring apis provide a generic single- or multi-tail ringbuffer... mirror
   protocol now uses this.  Features include ring elements may be sized to fit
   structs in the ringbuffer, callback when no tail any longer needs an element
   and it can be deleted, and zerocopy options to write new members directly
   into the ringbuffer, and use the ringbuffer element by address too.
 
 - abstract ssh 2 server plugin included, with both plugin and standalone
   demos provided.  You can bind the plugin to a vhost and also serve full-
   strength ssh from the vhost.  IO from the ssh server is controlled by an
   "ops" struct of callbacks for tx, rx, auth etc.
   
 - Many fixes, cleanups, source refactors and other improvements.


v2.3.0
======

 - ESP32 OpenSSL support for client and server

 - ESP32 4 x WLAN credential slots may be configured

 - Libevent event loop support

 - SOCKS5 proxy support

 - lws_meta protocol for websocket connection multiplexing

 - lws_vhost_destroy() added... allows dynamic removal of listening
   vhosts.  Vhosts with shared listen sockets adopt the listen socket
   automatically if the owner is destroyed.

 - IPv6 on Windows

 - Improved CGI handling suitable for general CGI scripting, eg, PHP

 - Convert even the "old style" test servers to use statically included
   plugin sources

 - LWS_WITH_STATS cmake option dumps resource usage and timing information
   every few seconds to debug log, including latency information about
   delay from asking for writeable callback to getting it

 - Large (> 2GB) files may be served

 - LWS_WITH_HTTP_PROXY Cmake option adds proxying mounts

 - Workaround for libev build by disabling -Werror on the test app

 - HTTP2 support disabled since no way to serve websockets on it


v2.2.0
======

Major new features

 - A mount can be protected by Basic Auth... in lwsws it looks like this

 ```
{
        "mountpoint": "/basic-auth",
        "origin": "file://_lws_ddir_/libwebsockets-test-server/private",
        "basic-auth": "/var/www/balogins-private"
}
```

The text file named in `basic-auth` contains user:password information
one per line.

See README.lwsws.md for more information.

 - RFC7233 RANGES support in lws server... both single and multipart.
 This allows seeking for multimedia file serving and download resume.
 It's enabled by default but can be disabled by CMake option.

 - On Linux, lwsws can reload configuration without dropping ongoing
 connections, when sent a SIGHUP.  The old configuration drops its
 listen sockets so the new configuration can listen on them.
 New connections connect to the server instance with the new
 configuration.  When all old connections eventually close, the old
 instance automatically exits.  This is equivalent to
 `systemctl reload apache`

 - New `adopt` api allow adoption including SSL negotiation and
 for raw sockets and file descriptors.

 - Chunked transfer encoding supported for client and server

 - Adaptations to allow operations inside OPTEE Secure World

 - ESP32 initial port - able to do all test server functions. See
 README.build.md

 - Serving gzipped files from inside a ZIP file is supported... this
 includes directly serving the gzipped content if the client
 indicated it could accept it (ie, almost all browsers) saving
 bandwidth and time.  For clients that can't accept it, lws
 automatically decompresses and serves the content in memory-
 efficient chunks. Only a few hundred bytes of heap are needed
 to serve any size file from inside the zip.  See README.coding.md

 - RAW file descriptors may now be adopted into the lws event loop,
 independent of event backend (including poll service).
 See README.coding.md

 - RAW server socket descriptors may now be enabled on the vhost if
 the first thing sent on the connection is not a valid http method.
 The user code can associate these with a specific protocol per
 vhost, and RAW-specific callbacks appear there for creation, rx,
 writable and close.  See libwebsockets-test-server-v2.0 for an example.
 See README.coding.md

 - RAW client connections are now possible using the method "RAW".
 After connection, the socket is associated to the protocol
 named in the client connection info and RAW-specific callbacks
 appear there for creation, rx, writable and close.
 See libwebsockets-test-client (with raw://) for an example.
 See README.coding.md


v2.1.0
======

Major new features

 - Support POST arguments, including multipart and file attachment

 - Move most of lwsws into lws, make the stub CC0

 - Add loopback test plugin to confirm client ws / http coexistence

 - Integrate lwsws testing on Appveyor (ie, windows)

 - Introduce helpers for sql, urlencode and urldecode sanitation

 - Introduce LWS_CALLBACK_HTTP_BIND_PROTOCOL / DROP_PROTOCOL that
 are compatible with http:/1.1 pipelining and different plugins
 owning different parts of the URL space

 - lwsgs - Generic Sessions plugin supports serverside sessions,
 cookies, hashed logins, forgot password etc

 - Added APIs for sending email to SMTP servers

 - Messageboard example plugin for lwsgs

 - Automatic PING sending at fixed intervals and close if no response

 - Change default header limit in ah to 4096 (from 1024)

 - Add SNI matching for wildcards if no specific wildcard vhost name match

 - Convert docs to Doxygen

 - ESP8266 support ^^

Fixes
-----

See git log v2.0.0..



v2.0.0
======

Summary
-------

 - There are only api additions, the api is compatible with v1.7.x.  But
   there is necessarily an soname bump to 8.
 
 - If you are using lws client, you mainly need to be aware the option
   LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT is needed at context-creation time
   if you will use SSL.
   
 - If you are using lws for serving, the above is also true but there are
   many new features to simplify your code (and life).  There is a
   summany online here
   
     https://libwebsockets.org/lws-2.0-new-features.html
     
   but basically the keywords are vhosts, mounts and plugins.  You can now
   do the web serving part from lws without any user callback code at all.
   See ./test-server/test-server-v2.0.c for an example, it has no user
   code for ws either since it uses the protocol plugins... that one C file
   is all that is needed to do the whole test server function.
   
   You now have the option to use a small generic ws-capable webserver
   "lwsws" and write your ws part as a plugin.  That eliminates even
   cut-and-pasting the test server code and offers more configurable
   features like control over http cacheability in JSON.


Fixes
-----

These are already in 1.7.x series

1) MAJOR (Windows-only) fix assert firing

2) MAJOR http:/1.1 connections handled by  lws_return_http_status() did not
get sent a content-length resulting in the link hanging until the peer closed
it.  attack.sh updated to add a test for this.

3) MINOR An error about hdr struct in _lws_ws_related is corrected, it's not
known to affect anything until after it was fixed

4) MINOR During the close shutdown wait state introduced at v1.7, if something
requests callback on writeable for the socket it will busywait until the
socket closes

5) MAJOR Although the test server has done it for a few versions already, it
is now required for the user code to explicitly call

	if (lws_http_transaction_completed(wsi))
		return -1;

when it finishes replying to a transaction in http.  Previously the library
did it for you, but that disallowed large, long transfers with multiple
trips around the event loop (and cgi...).

6) MAJOR connections on ah waiting list that closed did not get removed from
the waiting list...

7) MAJOR since we added the ability to hold an ah across http keepalive
transactions where more headers had already arrived, we broke the ability
to tell if more headers had arrived.  Result was if the browser didn't
close the keepalive, we retained ah for the lifetime of the keepalive,
using up the pool.

8) MAJOR windows-only-POLLHUP was not coming

9) Client should not send ext hdr if no exts

Changes
-------

1) MINOR test-server gained some new switches

   -C <file>  use external SSL cert file
   -K <file>  use external SSL key file
   -A <file>  use external SSL CA cert file
   
   -u <uid>  set effective uid
   -g <gid>  set effective gid

together you can use them like this to have the test-server work with the
usual purchased SSL certs from an official CA.

   --ssl -C your.crt -K your.key -A your.cer -u 99 -g 99

2) MINOR the OpenSSL magic to setup ECDH cipher usage is implemented in the
library, and the ciphers restricted to use ECDH only.
Using this, the lws test server can score an A at SSLLABS test

3) MINOR STS (SSL always) header is added to the test server if you use --ssl.  With
that, we score A+ at SSLLABS test

4) MINOR daemonize function (disabled at cmake by default) is updated to work
with systemd

5) MINOR example systemd .service file now provided for test server
(not installed by default)

6) test server html is updated with tabs and a new live server monitoring
feature.  Input sanitization added to the js.

7) client connections attempted when no ah is free no longer fail, they are
just deferred until an ah becomes available.

8) The test client pays attention to if you give it an http:/ or https://
protocol string to its argument in URL format.  If so, it stays in http[s]
client mode and doesn't upgrade to ws[s], allowing you to do generic http client
operations.  Receiving transfer-encoding: chunked is supported.

9) If you enable -DLWS_WITH_HTTP_PROXY=1 at cmake, the test server has a
new URI path http://localhost:7681/proxytest If you visit here, a client
connection to http://example.com:80 is spawned, and the results piped on
to your original connection.

10) Also with LWS_WITH_HTTP_PROXY enabled at cmake, lws wants to link to an
additional library, "libhubbub".  This allows lws to do html rewriting on the
fly, adjusting proxied urls in a lightweight and fast way.

11) There's a new context creation flag LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT,
this is included automatically if you give any other SSL-related option flag.
If you give no SSL-related option flag, nor this one directly, then even
though SSL support may be compiled in, it is never initialized nor used for the
whole lifetime of the lws context.

Conversely in order to prepare the context to use SSL, even though, eg, you
are not listening on SSL but will use SSL client connections later, you must
give this flag explicitly to make sure SSL is initialized.


User API additions
------------------

1) MINOR APIBREAK There's a new member in struct lws_context_creation_info, ecdh_curve,
which lets you set the name of the ECDH curve OpenSSL should use.  By
default (if you leave ecdh_curve NULL) it will use "prime256v1"

2) MINOR NEWAPI It was already possible to adopt a foreign socket that had not
been read from using lws_adopt_socket() since v1.7.  Now you can adopt a
partially-used socket if you don't need SSL, by passing it what you read
so it can drain that before reading from the socket.

LWS_VISIBLE LWS_EXTERN struct lws *
lws_adopt_socket_readbuf(struct lws_context *context, lws_sockfd_type accept_fd,
		const char *readbuf, size_t len);

3) MINOR NEWAPI CGI type "network io" subprocess execution is now possible from
a simple api.

LWS_VISIBLE LWS_EXTERN int
lws_cgi(struct lws *wsi, char * const *exec_array,  int script_uri_path_len,
        int timeout_secs);

LWS_VISIBLE LWS_EXTERN int
lws_cgi_kill(struct lws *wsi);

To use it, you must first set the cmake option

$ cmake .. -DLWS_WITH_CGI=1

See test-server-http.c and  test server path

http://localhost:7681/cgitest

stdin gets http body, you can test it with wget

$ echo hello > hello.txt
$ wget http://localhost:7681/cgitest --post-file=hello.txt -O- --quiet
lwstest script
read="hello"

The test script returns text/html table showing /proc/meminfo.  But the cgi
support is complete enough to run cgit cgi.

4) There is a helper api for forming logging timestamps

LWS_VISIBLE int
lwsl_timestamp(int level, char *p, int len)

this generates this kind of timestamp for use as logging preamble

lwsts[13116]: [2016/01/25 14:52:52:8386] NOTICE: Initial logging level 7

5) struct lws_client_connect_info has a new member

 const char *method
 
If it's NULL, then everything happens as before, lws_client_connect_via_info()
makes a ws or wss connection to the address given.

If you set method to a valid http method like "GET", though, then this method
is used and the connection remains in http[s], it's not upgraded to ws[s].

So with this, you can perform http[s] client operations as well as ws[s] ones.

There are 4 new related callbacks

	LWS_CALLBACK_ESTABLISHED_CLIENT_HTTP			= 44,
	LWS_CALLBACK_CLOSED_CLIENT_HTTP				= 45,
	LWS_CALLBACK_RECEIVE_CLIENT_HTTP			= 46,
	LWS_CALLBACK_COMPLETED_CLIENT_HTTP			= 47,

6) struct lws_client_connect_info has a new member

 const char *parent_wsi
 
if non-NULL, the client wsi is set to be a child of parent_wsi.  This ensures
if parent_wsi closes, then the client child is closed just before.

7) If you're using SSL, there's a new context creation-time option flag
LWS_SERVER_OPTION_REDIRECT_HTTP_TO_HTTPS.  If you give this, non-ssl
connections to the server listen port are accepted and receive a 301
redirect to / on the same host and port using https://

8) User code may set per-connection extension options now, using a new api
"lws_set_extension_option()".

This should be called from the ESTABLISHED callback like this

 lws_set_extension_option(wsi, "permessage-deflate",
                          "rx_buf_size", "12"); /* 1 << 12 */

If the extension is not active (missing or not negotiated for the
connection, or extensions are disabled on the library) the call is
just returns -1.  Otherwise the connection's extension has its
named option changed.

The extension may decide to alter or disallow the change, in the
example above permessage-deflate restricts the size of his rx
output buffer also considering the protocol's rx_buf_size member.


New application lwsws
---------------------

A libwebsockets-based general webserver is built by default now, lwsws.

It's configured by JSON, by default in

  /etc/lwsws/conf

which contains global lws context settings like this

{
  "global": {
   "uid": "99",
   "gid": "99",
   "interface": "eth0",
   "count-threads": "1"
 }
}

  /etc/lwsws/conf.d/*

which contains zero or more files describing vhosts, like this

{
 "vhosts": [
  { "name": "warmcat.com",
    "port": "443",
    "host-ssl-key": "/etc/pki/tls/private/warmcat.com.key",
    "host-ssl-cert": "/etc/pki/tls/certs/warmcat.com.crt",
    "host-ssl-ca": "/etc/pki/tls/certs/warmcat.com.cer",
    "mounts": [
      { "/": [
       { "home": "file:///var/www/warmcat.com" },
       { "default": "index.html" }
      ]
     }
    ]
   }
 ]
}



v1.7.0
======

Extension Changes
-----------------

1) There is now a "permessage-deflate" / RFC7692 implementation.  It's very
similar to "deflate-frame" we have offered for a long while; deflate-frame is
now provided as an alias of permessage-deflate.

The main differences are that the new permessage-deflate implementation:

 - properly performs streaming respecting input and output buffer limits.  The
   old deflate-frame implementation could only work on complete deflate input
   and produce complete inflate output for each frame.  The new implementation
   only mallocs buffers at initialization.

 - goes around the event loop after each input package is processed allowing
   interleaved output processing.  The RX flow control api can be used to
   force compressed input processing to match the rate of compressed output
   processing (test--echo shows an example of how to do this).

 - when being "deflate-frame" for compatibility he uses the same default zlib
   settings as the old "deflate-frame", but instead of exponentially increasing
   malloc allocations until the whole output will fit, he observes the default
   input and output chunking buffer sizes of "permessage-deflate", that's
   1024 in and 1024 out at a time.

2) deflate-stream has been disabled for many versions (for over a year) and is
now removed.  Browsers are now standardizing on "permessage-deflate" / RFC7692

3) struct lws_extension is simplified, and lws extensions now have a public
api (their callback) for use in user code to compose extensions and options
the user code wants.  lws_get_internal_exts() is deprecated but kept around
as a NOP.  The changes allow one extension implementation to go by different
names and allows the user client code to control option offers per-ext.

The test client and server are updated to use the new way.  If you use
the old way it should still work, but extensions will be disabled until you
update your code.

Extensions are now responsible for allocating and per-instance private struct
at instance construction time and freeing it when the instance is destroyed.
Not needing to know the size means the extension's struct can be opaque
to user code.


User api additions
------------------

1) The info struct gained three new members

 - max_http_header_data: 0 for default (1024) or set the maximum amount of known
    http header payload that lws can deal with.  Payload in unknown http
    headers is dropped silently.  If for some reason you need to send huge
    cookies or other HTTP-level headers, you can now increase this at context-
    creation time.

 - max_http_header_pool: 0 for default (16) or set the maximum amount of http
     headers that can be tracked by lws in this context.  For the server, if
     the header pool is completely in use then accepts on the listen socket
     are disabled until one becomes free.  For the client, if you simultaneously
     have pending connects for more than this number of client connections,
     additional connects will fail until some of the pending connections timeout
     or complete.

 - timeout_secs: 0 for default (currently 20s), or set the library's
     network activity timeout to the given number of seconds

HTTP header processing in lws only exists until just after the first main
callback after the HTTP handshake... for ws connections that is ESTABLISHED and
for HTTP connections the HTTP callback.

So these settings are not related to the maximum number of simultaneous
connections, but the number of HTTP handshakes that may be expected or ongoing,
or have just completed, at one time.  The reason it's useful is it changes the
memory allocation for header processing to be one-time at context creation
instead of every time there is a new connection, and gives you control over
the peak allocation.

Setting max_http_header_pool to 1 is fine it will just queue incoming
connections before the accept as necessary, you can still have as many
simultaneous post-header connections as you like.  Since the http header
processing is completed and the allocation released after ESTABLISHED or the
HTTP callback, even with a pool of 1 many connections can be handled rapidly.

2) There is a new callback that allows the user code to get acccess to the
optional close code + aux data that may have been sent by the peer.

LWS_CALLBACK_WS_PEER_INITIATED_CLOSE:
             The peer has sent an unsolicited Close WS packet.  @in and
             @len are the optional close code (first 2 bytes, network
             order) and the optional additional information which is not
             defined in the standard, and may be a string or non-human-
             readble data.
             If you return 0 lws will echo the close and then close the
             connection.  If you return nonzero lws will just close the
             connection.

As usual not handling it does the right thing, if you're not interested in it
just ignore it.

The test server has "open and close" testing buttons at the bottom, if you
open and close that connection, on close it will send a close code 3000 decimal
and the string "Bye!" as the aux data.

The test server dumb-increment callback handles this callback reason and prints

lwsts[15714]: LWS_CALLBACK_WS_PEER_INITIATED_CLOSE: len 6
lwsts[15714]:  0: 0x0B
lwsts[15714]:  1: 0xB8
lwsts[15714]:  2: 0x42
lwsts[15714]:  3: 0x79
lwsts[15714]:  4: 0x65
lwsts[15714]:  5: 0x21

3) There is a new API to allow the user code to control the content of the
close frame sent when about to return nonzero from the user callback to
indicate the connection should close.

/**
 * lws_close_reason - Set reason and aux data to send with Close packet
 *		If you are going to return nonzero from the callback
 *		requesting the connection to close, you can optionally
 *		call this to set the reason the peer will be told if
 *		possible.
 *
 * @wsi:	The websocket connection to set the close reason on
 * @status:	A valid close status from websocket standard
 * @buf:	NULL or buffer containing up to 124 bytes of auxiliary data
 * @len:	Length of data in @buf to send
 */
LWS_VISIBLE LWS_EXTERN void
lws_close_reason(struct lws *wsi, enum lws_close_status status,
		 unsigned char *buf, size_t len);

An extra button is added to the "open and close" test server page that requests
that the test server close the connection from his end.

The test server code will do so by

			lws_close_reason(wsi, LWS_CLOSE_STATUS_GOINGAWAY,
					 (unsigned char *)"seeya", 5);
			return -1;

The browser shows the close code and reason he received

websocket connection CLOSED, code: 1001, reason: seeya

4) There's a new context creation time option flag

LWS_SERVER_OPTION_VALIDATE_UTF8

if you set it in info->options, then TEXT and CLOSE frames will get checked to
confirm that they contain valid UTF-8.  If they don't, the connection will get
closed by lws.

5) ECDH Certs are now supported.  Enable the CMake option

cmake .. -DLWS_SSL_SERVER_WITH_ECDH_CERT=1 

**and** the info->options flag

LWS_SERVER_OPTION_SSL_ECDH

to build in support and select it at runtime.

6) There's a new api lws_parse_uri() that simplifies chopping up
https://xxx:yyy/zzz uris into parts nicely.  The test client now uses this
to allow proper uris as well as the old address style.

7) SMP support is integrated into LWS without any internal threading.  It's
very simple to use, libwebsockets-test-server-pthread shows how to do it,
use -j <n> argument there to control the number of service threads up to 32.

Two new members are added to the info struct

	unsigned int count_threads;
	unsigned int fd_limit_per_thread;
	
leave them at the default 0 to get the normal singlethreaded service loop.

Set count_threads to n to tell lws you will have n simultaneous service threads
operating on the context.

There is still a single listen socket on one port, no matter how many
service threads.

When a connection is made, it is accepted by the service thread with the least
connections active to perform load balancing.

The user code is responsible for spawning n threads running the service loop
associated to a specific tsi (Thread Service Index, 0 .. n - 1).  See
the libwebsockets-test-server-pthread for how to do.

If you leave fd_limit_per_thread at 0, then the process limit of fds is shared
between the service threads; if you process was allowed 1024 fds overall then
each thread is limited to 1024 / n.

You can set fd_limit_per_thread to a nonzero number to control this manually, eg
the overall supported fd limit is less than the process allowance.

You can control the context basic data allocation for multithreading from Cmake
using -DLWS_MAX_SMP=, if not given it's set to 32.  The serv_buf allocation
for the threads (currently 4096) is made at runtime only for active threads.

Because lws will limit the requested number of actual threads supported
according to LWS_MAX_SMP, there is an api lws_get_count_threads(context) to
discover how many threads were actually allowed when the context was created.

It's required to implement locking in the user code in the same way that
libwebsockets-test-server-pthread does it, for the FD locking callbacks.

If LWS_MAX_SMP=1, then there is no code related to pthreads compiled in the
library.  If more than 1, a small amount of pthread mutex code is built into
the library.

8) New API

LWS_VISIBLE struct lws *
lws_adopt_socket(struct lws_context *context, lws_sockfd_type accept_fd)

allows foreign sockets accepted by non-lws code to be adopted by lws as if they
had just been accepted by lws' own listen socket.

9) X-Real-IP: header has been added as WSI_TOKEN_HTTP_X_REAL_IP

10) Libuv support is added, there are new related user apis

typedef void (lws_uv_signal_cb_t)(uv_loop_t *l, uv_signal_t *w, int revents);

LWS_VISIBLE LWS_EXTERN int
lws_uv_sigint_cfg(struct lws_context *context, int use_uv_sigint,
		  lws_uv_signal_cb_t *cb);

LWS_VISIBLE LWS_EXTERN int
lws_uv_initloop(struct lws_context *context, uv_loop_t *loop, int tsi);

LWS_VISIBLE void
lws_uv_sigint_cb(uv_loop_t *loop, uv_signal_t *watcher, int revents);

and CMAKE option

LWS_WITH_LIBUV


User api changes
----------------

1) LWS_SEND_BUFFER_POST_PADDING is now 0 and deprecated.  You can remove it; if
you still use it, obviously it does nothing.  Old binary code with nonzero
LWS_SEND_BUFFER_POST_PADDING is perfectly compatible, the old code just
allocated a buffer bigger than the library is going to use.

The example apps no longer use LWS_SEND_BUFFER_POST_PADDING.

The only path who made use of it was sending with LWS_WRITE_CLOSE --->

2) Because of lws_close_reason() formalizing handling close frames,
LWS_WRITE_CLOSE is removed from libwebsockets.h.  It was only of use to send
close frames...close frame content should be managed using lws_close_reason()
now.

3) We check for invalid CLOSE codes and complain about protocol violation in
our close code.  But it changes little since we were in the middle of closing
anyway.

4) zero-length RX frames and zero length TX frames are now allowed.