To prevent one subsystem's taskprocessors from causing others
to stall, new capabilities have been added to taskprocessors.

* Any taskprocessor name that has a '/' will have the part
  before the '/' saved as its "subsystem".
  Examples:
  "sorcery/acl-0000006a" and "sorcery/aor-00000019"
  will be grouped to subsystem "sorcery".
  "pjsip/distributor-00000025" and "pjsip/distributor-00000026"
  will bn grouped to subsystem "pjsip".
  Taskprocessors with no '/' have an empty subsystem.

* When a taskprocessor enters high-water alert status and it
  has a non-empty subsystem, the subsystem alert count will
  be incremented.

* When a taskprocessor leaves high-water alert status and it
  has a non-empty subsystem, the subsystem alert count will be
  decremented.

* A new api ast_taskprocessor_get_subsystem_alert() has been
  added that returns the number of taskprocessors in alert for
  the subsystem.

* A new CLI command "core show taskprocessor alerted subsystems"
  has been added.

* A new unit test was addded.

REMINDER: The taskprocessor code itself doesn't take any action
based on high-water alerts or overloading.  It's up to taskprocessor
users to check and take action themselves.  Currently only the pjsip
distributor does this.

* A new pjsip/global option "taskprocessor_overload_trigger"
  has been added that allows the user to select the trigger
  mechanism the distributor uses to pause accepting new requests.
  "none": Don't pause on any overload condition.
  "global": Pause on ANY taskprocessor overload (the default and
  current behavior)
  "pjsip_only": Pause only on pjsip taskprocessor overloads.

* The core pjsip pool was renamed from "SIP" to "pjsip" so it can
  be properly grouped into the "pjsip" subsystem.

* stasis taskprocessor names were changed to "stasis" as the
  subsystem.

* Sorcery core taskprocessor names were changed to "sorcery" to
  match the object taskprocessors.

Change-Id: I8c19068bb2fc26610a9f0b8624bdf577a04fcd56

The commit I2f97ebfa79969a36a97bb7b9afd5b6268cf1a07d removed sending out
the ContactStatus AMI event when a contact is updated.
Thist change broke things which rely on old behavior.

This patch adds a new PJSIP global configuration option
'send_contact_status_on_update_registration' to be able to preserve old
ContactStatus behavior.
By default new behavior, i.e. the ContactStatus event will not be sent when a
device refreshes its registration.

Change-Id: I706adf7584e7077eb6bde6d9799ca408bc82ce46

The use of a '|' in the "global/debug" synopsis documentation caused the
generated html table on the wiki to add an extra column that included the
text after the pipe.

This patch replaces the pipe with a comma.

ASTERISK-28150

Change-Id: I3d79a6ca6d733d9cb290e779438114884b98a719

The current round-robin method does not take the current taskprocessor
load into consideration when distributing requests.  Using the least-size
method the request goes to the taskprocessor that is servicing the least
number of active tasks at the current time.

Longer running tasks with the round-robin method can delay processing
tasks.

* Change the algorithm from round-robin to least-size for picking the
PJSIP taskprocessor from the default serializer pool.

Change-Id: I7b8d8cc2c2490494f579374b6af0a4868e3a37cd

ASTERISK-28087

Change-Id: I69d48813ec514f5ef06c6de994cba52630e0a3b4

This patch adds new options 'trust_connected_line' and 'send_connected_line'
to the endpoint.

The option 'trust_connected_line' is to control if connected line updates
are accepted from this endpoint.

The option 'send_connected_line' is to control if connected line updates
can be sent to this endpoint.

The default value is 'yes' for both options.

Change-Id: I16af967815efd904597ec2f033337e4333d097cd

This change implements a few different generic things which were brought
on by Google Voice SIP.

1.  The concept of flow transports have been introduced.  These are
configurable transports in pjsip.conf which can be used to reference a
flow of signaling to a target.  These have runtime configuration that can
be changed by the signaling itself (such as Service-Routes and
P-Preferred-Identity).  When used these guarantee an individual connection
(in the case of TCP or TLS) even if multiple flow transports exist to the
same target.

2.  Service-Routes (RFC 3608) support has been added to the outbound
registration module which when received will be stored on the flow
transport and used for requests referencing it.

3.  P-Associated-URI / P-Preferred-Identity (RFC 3325) support has been
added to the outbound registration module.  If a P-Associated-URI header
is received it will be used on requests as the P-Preferred-Identity.

4.  Configurable outbound extension support has been added to the outbound
registration module.  When set the extension will be placed in the
Supported header.

5.  Header parameters can now be configured on an outbound registration
which will be placed in the Contact header.

6.  Google specific OAuth / Bearer token authentication
(draft-ietf-sipcore-sip-authn-02) has been added to the outbound
registration module.

All functionality changes are controlled by pjsip.conf configuration
options and do not affect non-configured pjsip endpoints otherwise.

ASTERISK-27971 #close

Change-Id: Id214c2d1c550a41fcf564b7df8f3da7be565bd58

Both pjsip_tx_data.tp_info.dst_name and pjsip_rx_data.pkt_info.src_name
store IPv6 addresses without enclosing brackets. This causes some log
output to be confusing because it is difficult to separate the IPv6
address from a port specification.

* Use pj_sockaddr_print() along with pjsip_tx_data.tp_info.dst_addr and
  pjsip_rx_data.pkt_info.src_addr where possible for consistent IPv6
  output.

* When a pj_sockaddr is not available, explicitly wrap IPv6 addresses
  in brackets.

* When assigning pjsip_rx_data.pkt_info.src_name ourselves, make sure
  to also set pjsip_rx_data.pkt_info.src_addr.

Change-Id: I5cfe997ced7883862a12b9c7d8551d76ae02fcf8

If a SIP MESSAGE is triggered for an endpoint that is currently not registered
- and therefore has no valid contact associated - an error message was logged.
Since this is a valid request in a valid use cases this is now changed to a
warning, as discussed with Matt Fredrickson on the asterisk-dev mailing list.

Change-Id: I55eb62d2712818a58c7532119dec288bd98cf0c0

A change recently went in which disabled the built-in PJSIP
keepalive. This defaulted to 90 seconds and kept TCP/TLS
connections alive. Disabling this functionality has resulted
in a behavior change of not doing keepalives by default resulting
in TCP/TLS connections dropping for some people.

This change makes our default keepalive interval 90 seconds
to match the previous behavior and preserve it.

ASTERISK-27978

Change-Id: Ibd9a45f3cbe5d9bb6d2161268696645ff781b1d6

Change-Id: I5394fdff6a296efc8e1695a156e616acd932ae52

A problem I've seen countless times is a global or system section
for PJSIP not getting applied. This is inevitably the result of
the "type=" line missing. This change alleviates that problem.

The ability to specify an explicit section name has been
added to res_sorcery_config. If the configured section
name matches this and there are no unknown things configured
the section is taken as being for the given type.

Both the PJSIP "global" and "system" types now support this
so you can just name your section "global" or "system" and it
will be matched and used, even without a "type=" line.

ASTERISK-27972

Change-Id: Ie22723663c1ddd24f869af8c9b4c1b59e2476893

A new option 'suppress_q850_reason_headers' has been added to the
endpoint object. Some devices can't accept multiple Reason headers and
get confused when both 'SIP' and 'Q.850' Reason headers are received.
This option allows the 'Q.850' Reason header to be suppressed.
The default value is 'no'.

ASTERISK-27949
Reported-by: Ross Beer

Change-Id: I54cf37a827d77de2079256bb3de7e90fa5e1deb1

The Websocket transport uses the built-in HTTP server. As a result
the TLS configuration is done in http.conf and not in pjsip.conf.

This change adds a warning if this is configured in pjsip.conf and
also clarifies in the sample configuration file.

Change-Id: I187d994d328c3ed274b6754fd4c2a4955bdc6dd9

pjproject by default currently will follow media forked during an INVITE
on outbound calls if the To tag is different on a subsequent response as
that on an earlier response.  We handle this correctly.  There have
been reported cases where the To tag is the same but we still need to
follow the media.  The pjproject patch in this commit adds the
capability to sip_inv and also adds the capability to control it at
runtime.  The original "different tag" behavior was always controllable
at runtime but we never did anything with it and left it to default to
TRUE.

So, along with the pjproject patch, this commit adds options to both the
system and endpoint objects to control the two behaviors, and a small
logic change to session_inv_on_media_update in res_pjsip_session to
control the behavior at the endpoint level.

The default behavior for "different tags" remains the same at TRUE and
the default for "same tag" is FALSE.

Change-Id: I64d071942b79adb2f0a4e13137389b19404fe3d6
ASTERISK-27936
Reported-by: Ross Beer

The OPTIONS support in PJSIP has organically grown, like many things in
Asterisk.  It has been tweaked, changed, and adapted based on situations
run into.  Unfortunately this has taken its toll.  Configuration file
based objects have poor performance and even dynamic ones aren't that
great.

This change scraps the existing code and starts fresh with new eyes.  It
leverages all of the APIs made available such as sorcery observers and
serializers to provide a better implementation.

1.  The state of contacts, AORs, and endpoints relevant to the qualify
process is maintained.  This state can be updated by external forces (such
as a device registering/unregistering) and also the reload process.  This
state also includes the association between endpoints and AORs.

2.  AORs are scheduled and not contacts.  This reduces the amount of work
spent juggling scheduled items.

3.  Manipulation of which AORs are being qualified and the endpoint states
all occur within a serializer to reduce the conflict that can occur with
multiple threads attempting to modify things.

4.  Operations regarding an AOR use a serializer specific to that AOR.

5.  AORs and endpoint state act as state compositors.  They take input
from lower level objects (contacts feed AORs, AORs feed endpoint state)
and determine if a sufficient enough change has occurred to be fed further
up the chain.

6.  Realtime is supported by using observers to know when a contact has
been registered.  If state does not exist for the associated AOR then it
is retrieved and becomes active as appropriate.

The end result of all of this is best shown with a configuration file of
3000 endpoints each with an AOR that has a static contact.  In the old
code it would take over a minute to load and use all 8 of my cores.  This
new code takes 2-3 seconds and barely touches the CPU even while dealing
with all of the OPTIONS requests.

ASTERISK-26806

Change-Id: I6a5ebbfca9001dfe933eaeac4d3babd8d2e6f082

ast_sip_push_task_synchronous() did not necessarily execute the passed in
task under the specified serializer.  If the current thread is any
registered pjsip thread then it would execute the task immediately instead
of under the specified serializer.  Reentrancy issues could result if the
task does not execute with the right serializer.

The original reason ast_sip_push_task_synchronous() checked to see if the
current thread was a registered pjsip thread was because of a deadlock
with masquerades and the channel technology's fixup callback
(ASTERISK_22936).  A subsequent masquerade deadlock fix (ASTERISK_24356)
involving call pickups avoided the original deadlock situation entirely.
The PJSIP channel technology's fixup callback no longer needed to call
ast_sip_push_task_synchronous().

However, there are a few places where this unexpected behavior is still
required to avoid deadlocks.  The pjsip monitor thread executes callbacks
that do calls to ast_sip_push_task_synchronous() that would deadlock if
the task were actually pushed to the specified serializer.  I ran into one
dealing with the pubsub subscriptions where an ao2 destructor called
ast_sip_push_task_synchronous().

* Split ast_sip_push_task_synchronous() into
ast_sip_push_task_wait_servant() and ast_sip_push_task_wait_serializer().
ast_sip_push_task_wait_servant() has the old behavior of
ast_sip_push_task_synchronous().  ast_sip_push_task_wait_serializer() has
the new behavior where the task is always executed by the specified
serializer or a picked serializer if one is not passed in.  Both functions
behave the same if the current thread is not a SIP servant.

* Redirected ast_sip_push_task_synchronous() to
ast_sip_push_task_wait_servant() to preserve API for released branches.

ASTERISK_26806

Change-Id: Id040fa42c0e5972f4c8deef380921461d213b9f3

This patch adds support to send in-dialog SIP NOTIFY commands on
chan_pjsip channels, similar to the functionality recently added
for chan_sip (ASTERISK_27461).

This extends res_pjsip_notify to allow for in-dialog messages.

ASTERISK-27697

Change-Id: If7f3151a6d633e414d5dc319d5efc1443c43dd29

Change-Id: I3811de0014b1ffe96d4a3b49cddd5d4ca02ee5d4

* acl (named_acl.c)
* cdr
* cel
* ccss
* dnsmgr
* dsp
* enum
* extconfig (config.c)
* features
* http
* indications
* logger
* manager
* plc
* sounds
* udptl

These modules are now loaded at appropriate time by the module loader.
Unlike loadable modules these use AST_MODULE_LOAD_FAILURE on error so
the module loader will abort startup on failure of these modules.

Some of these modules are still initialized or shutdown from outside the
module loader.  logger.c is initialized very early and shutdown very
late, manager.c is initialized by the module loader but is shutdown by
the Asterisk core (too much uses it without holding references).

Change-Id: I371a9a45064f20026c492623ea8062d02a1ab97f

The pool cache gets in the way of finding use after free errors of memory
pool contents.  Tools like valgrind and MALLOC_DEBUG don't know when a
pool is released because it gets put into the cache instead of being
freed.

* Added the "cache_pools" option to pjproject.conf.  Disabling the option
helps track down pool content mismanagement when using valgrind or
MALLOC_DEBUG.  The cache gets in the way of determining if the pool
contents are used after free and who freed it.

To disable the pool caching simply disable the cache_pools option in
pjproject.conf and restart Asterisk.

Sample pjproject.conf setting:
[startup]
cache_pools=no

* Made current users of the caching pool factory initialization and
destruction calls call common routines to create and destroy cached pools.

ASTERISK-27704

Change-Id: I64d5befbaeed2532f93aa027a51eb52347d2b828

Since res_pjsip_transport_management provides several attack
mitigation features, its functionality moved to res_pjsip and
this module has been removed.  This way the features will always
be available if res_pjsip is loaded.

ASTERISK-27618
Reported By: Sandro Gauci

Change-Id: I21a2d33d9dda001452ea040d350d7a075f9acf0d

pjsip_distributor:
   authenticate() creates a tdata and uses it to send a challenge or
   failure response.  When pjsip_endpt_send_response2() succeeds, it
   automatically decrements the tdata ref count but when it fails, it
   doesn't.  Since we weren't checking for a return status, we weren't
   decrementing the count ourselves on error and were therefore leaking
   tdatas.

res_pjsip_session:
   session_reinvite_on_rx_request wasn't decrementing the ref count
   if an error happened while sending a 491 response.
   pre_session_setup wasn't decrementing the ref count if
   while sending an error after a pjsip_inv_verify_request failure.

res_pjsip:
   ast_sip_send_response wasn't decrementing the ref count on error.

ASTERISK-27618
Reported By: Sandro Gauci

Change-Id: Iab33a6c7b6fba96148ed465b690ba8534ac961bf

It was discovered that there are some corner cases where a pjsip tsx
might have no last_tx so calling ast_sip_failover_request with
a NULL last_tx as its tdata would cause a crash.

ASTERISK-27618
Reported By:  Sandro Gauci

Change-Id: Ic2b63f6d4ae617c4c19dcdec2a7a6156b54fd15b

There is a dedicated slot in the pjsip_sip_uri for the 'user'
parameter, so use that instead of adding to the list of generic URI
parameters.

Change-Id: I0a0ce8a60ecee27489735bf56fd707719d8c2ed6

Change-Id: I82ae0b92bfa2ece84a5c684efd9eefdc83ebd068

In an earlier release, inbound registrations on a reliable transport
were pruned on Asterisk restart since the TCP connection would have
been torn down and become unusable when Asterisk stopped.  This same
process is now also applied to inbound subscriptions.

Also fixed issues in res_pjsip_registrar where it wasn't handling the
monitoring correctly when multiple registrations came in over the same
transport.

To accomplish this, the pjsip_transport_event feature needed to
be refactored to allow multiple monitors (multiple subcriptions or
registrations from the same endpoint) to exist on the same transport.
Since this changed the API, any external modules that may have used the
transport monitor feature (highly unlikey) will need to be changed.

ASTERISK-27612
Reported by: Ross Beer

Change-Id: Iee87cf4eb9b7b2b93d5739a72af52d6ca8fbbe36

This removes references that are no longer needed due to automatic
references created by module dependencies.

In addition this removes most calls to ast_module_check as they were
checking modules which are listed as dependencies.

Change-Id: I332a6e8383d4c72c8e89d988a184ab8320c4872e

Add an AMI action which provides information on all
configured Contacts.

ASTERISK-27581

Change-Id: I2eed42c74bbc725fad26b8b33b1a5b3161950c73

I've audited all modules that include any header which includes
asterisk/optional_api.h.  All modules which use OPTIONAL_API now declare
those dependencies in AST_MODULE_INFO using requires or optional_modules
as appropriate.

In addition ARI dependency declarations have been reworked.  Instead of
declaring additional required modules in res/ari/resource_*.c we now add
them to an optional array "requiresModules" in api-docs for each module.
This allows the AST_MODULE_INFO dependencies to include those missing
modules.

Change-Id: Ia0c70571f5566784f63605e78e1ceccb4f79c606

Change-Id: I67ed9039bf3f132fb20ee7a750e0aef0f704d7d3

The type=identify endpoint identification method can match by IP address
and by SIP header.  However, the SIP header matching has limited
usefulness because you cannot specify the SIP header matching priority
relative to the IP address matching.  All the matching happens at the same
priority and the order of evaluating the identify sections is
indeterminate.  e.g., If you had two type=identify sections where one
matches by IP address for endpoint alice and the other matches by SIP
header for endpoint bob then you couldn't predict which endpoint is
matched when a request comes in that matches both.

* Extract the SIP header matching criteria into its own "header" endpoint
identification method so the user can specify the relative priority of the
SIP header and the IP address matching criteria in the global
endpoint_identifier_order option.  The "ip" endpoint identification method
now only matches by IP address.

ASTERISK-27491

Change-Id: I9df142a575b7e1e3471b7cda5d3ea156cef08095

* Declare 'requires' and 'enhances' text fields on module info structure.
* Rename 'nonoptreq' to 'optional_modules'.
* Update doxygen comments.

Still need to investigate dependencies among modules I cannot compile.

Change-Id: I3ad9547a0a6442409ff4e352a6d897bef2cc04bf

* Endpoint identify_by documentation.
* IP/Header endpoint identifier documentation.

Change-Id: Id92f00b495acca7be945daf749d2abd7f76a0b5a

Add an AMI action which provides information on all
configured Auths.

ASTERISK-27547

Change-Id: I1a88a75b38a2b1dd9d1de6c0307b20a3f584c817

If an endpoint identifier name in the endpoint_identifier_order list is a
prefix to the identifier we are registering, we could install it in the
wrong position of the list.

Assuming
endpoint_identifier_order=username,ip,anonymous

then registering the "ip_only" identifier would put the identifier in the
wrong position of the priority list.

* Fix incorrect strncmp() string prefix matching.

Change-Id: Ib8819ec4b811da8a27419fd93528c54d34f01484

Add an AMI action which provides information on all
configured AORs.

ASTERISK-27537

Change-Id: If8b990a00909e5b6c0f04a3b8dccd9903dc445eb

Those SIP messages that create dialogs require a contact header to be present.
If the contact header was missing from the message it could cause Asterisk to
crash.

This patch checks to make sure SIP messages that create a dialog contain the
contact header. If the message does not and it is required Asterisk now returns
a "400 Missing Contact header" response. Also added NULL checks when retrieving
the contact header that were missing as a "just in case".

ASTERISK-27480 #close

Change-Id: I1810db87683fc637a9e3e1384a746037fec20afe

A couple of places were setting the status to "UNKNOWN" when qualifies were
being disabled. Instead this should be set to the "CREATED" status that
represents when a contact is given (uri available), but the qualify frequency
is set to zero so we don't know the status.

This patch updates the relevant places with "CREATED". It also updates the
"CREATED" status description (value shown in CLI/AMI/ARI output) to a value
of "NonQualified"/"NonQual" as this description is hopefully less confusing.

ASTERISK-27467

Change-Id: Id67509d25df92a72eb3683720ad2a95a27b50c89

This mimics the behavior of Chrome and Firefox and creates an ephemeral
X.509 certificate for each DTLS session.

Currently, the only supported key type is ECDSA because of its faster
generation time, but other key types can be added in the future as
necessary.

ASTERISK-27395

Change-Id: I5122e5f4b83c6320cc17407a187fcf491daf30b4