In res_pjsip_sdp_rtp, the bind_rtp_to_media_address option and the
fallback use of the transport's bind address solve problems sending
media on systems that cannot send ipv4 packets on ipv6 sockets, and
certain other situations. This change extends both of these behaviors
to UDPTL sessions as well in res_pjsip_t38, to fix fax-specific
problems on these systems, introducing a new option
endpoint/t38_bind_udptl_to_media_address.

ASTERISK-29402

Change-Id: I87220c0e9cdd2fe9d156846cb906debe08c63557

When Asterisk sends a reinvite negotiating T38 faxing, it's possible a
crash can occur if the response contains a m=image and zero port. The
reinvite callback code now checks session_media to see if it is null or
not before trying to access the udptl variable on it.

ASTERISK-29305

Change-Id: I1dfc51c5fa586e38579ede4bc228edee213ccaa9

When an endpoint requests to re-negotiate for fax and the incoming
re-invite is received prior to Asterisk sending out the 200 OK for
the initial invite the re-invite gets delayed. When Asterisk does
finally send the re-inivite the SDP includes streams for both audio
and T.38.

This happens because when the pending topology and active topologies
differ (pending stream is not in the active) in the delayed scenario
the pending stream is appended to the active topology. However, in
the fax case the pending stream should replace the active.

This patch makes it so when a delay occurs during fax negotiation,
to or from, the audio stream is replaced by the T.38 stream, or vice
versa instead of being appended.

Further when Asterisk sent the re-invite with both audio and T.38,
and the endpoint responded with a declined T.38 stream then Asterisk
would crash when attempting to change the T.38 state.

This patch also puts in a check that ensures the media state has a
valid fax session (associated udptl object) before changing the
T.38 state internally.

ASTERISK-29203 #close

Change-Id: I407f4fa58651255b6a9030d34fd6578cf65ccf09

if asterisk offer T38 SDP with none error correction scheme and
the endpoint respond with redundancy EC scheme, asterisk switch
to that mode. Since we configure the endpoint as none EC mode
we should not switch to any other mode except none.
following logic implemented in code.

1. If asterisk offer none, and anything except none in answer
   will be ignored.
2. If asterisk offer fec, answer with fec, redundancy and none will
   be accepted.
3. If asterisk offer redundancy, answer with redundancy and none
   will be accepted.

ASTERISK-28621

Change-Id: I343c62253ea4c8b7ee17abbfb377a4d484a14b19

This patch fixes several issues reported by the lgtm code analysis tool:

https://lgtm.com/projects/g/asterisk/asterisk

Not all reported issues were addressed in this patch. This patch mostly fixes
confirmed reported errors, potential problematic code points, and a few other
"low hanging" warnings or recommendations found in core supported modules.
These include, but are not limited to the following:

* innapropriate stack allocation in loops
* buffer overflows
* variable declaration "hiding" another variable declaration
* comparisons results that are always the same
* ambiguously signed bit-field members
* missing header guards

Change-Id: Id4a881686605d26c94ab5409bc70fcc21efacc25

After receiving a 200 OK with a declined stream in response to a T.38
initiated re-invite Asterisk would crash when attempting to dereference
a NULL session media object.

This patch checks to make sure the session media object is not NULL before
attempting to use it.

ASTERISK-28495
patches:
  ast-2019-004.patch submitted by Alexei Gradinari (license 5691)

Change-Id: I168f45f4da29cfe739acf87e597baa2aae7aa572

We were still getting crashes after the first fix.  Somehow we receive a
non-2xx final response before we get a 200 final response.  With the
failure response we had already cleaned up and destroyed some data
structures.  When the unexpected 200 response comes in we crash.

* Add protection code to prevent processing another final T.38 reINVITE
response.

ASTERISK-27944

Change-Id: I8b5baba8d07fe4d63f0d7d05d3eb9a3d27d40a74

When negotiating an incoming T.38 stream the code incorrectly
returned failure instead of a decline for the stream when a
problem occurred or the configuration didn't allow it. This
resulted in SDP offers being rejected with a 488 response
in all cases, even when another valid stream was present.

This change makes it so the stream is now declined. If no
streams are accepted a 488 response is sent while if at least
one stream is accepted all the declined streams are, well,
declined.

ASTERISK-27763

Change-Id: I88bcf793788c412a9839d111a5c736bf6867807c

We were blindly responding with AST_T38_REFUSED when ANY T.38 control
frame came accross the bridge.  This causes T.38 Gateway to get confused
and the T.38 session to get in a strange state.

* Made the T.38 framehook only respond to request frames and ignore
response frames.

ASTERISK-27657
ASTERISK-27080

Change-Id: I5fb5967c7d1efb30a7ff375f82887ca82a55b05b

If we initiated a T.38 reINVITE, we would crash if we received any other
1xx response message except 100 if it were followed by a 200 response.

* Made ignore any 1xx response so we do not close out the T.38 negotiation
too early.  For good measure we'll now accept any 2xx response as
acceptance of the reINVITE T.38 offer.

ASTERISK-27944

Change-Id: I0ca88aae708d091db7335af73f41035a212adff4

Fix some timer heap initializations and cancels to try and prevent
crashes and timer heap issues.

Change-Id: I64885d190fa22097d1b55987091375541e57a7ee

* acl (named_acl.c)
* cdr
* cel
* ccss
* dnsmgr
* dsp
* enum
* extconfig (config.c)
* features
* http
* indications
* logger
* manager
* plc
* sounds
* udptl

These modules are now loaded at appropriate time by the module loader.
Unlike loadable modules these use AST_MODULE_LOAD_FAILURE on error so
the module loader will abort startup on failure of these modules.

Some of these modules are still initialized or shutdown from outside the
module loader.  logger.c is initialized very early and shutdown very
late, manager.c is initialized by the module loader but is shutdown by
the Asterisk core (too much uses it without holding references).

Change-Id: I371a9a45064f20026c492623ea8062d02a1ab97f

This removes references that are no longer needed due to automatic
references created by module dependencies.

In addition this removes most calls to ast_module_check as they were
checking modules which are listed as dependencies.

Change-Id: I332a6e8383d4c72c8e89d988a184ab8320c4872e

* Declare 'requires' and 'enhances' text fields on module info structure.
* Rename 'nonoptreq' to 'optional_modules'.
* Update doxygen comments.

Still need to investigate dependencies among modules I cannot compile.

Change-Id: I3ad9547a0a6442409ff4e352a6d897bef2cc04bf

Change-Id: I81b2587427c6982aa3e2a3f9ad69cce8d316eb10

In 45744fc5, I mistakenly broke SDP media address rewriting by
misinterpreting which address was checked in the localnet comparison.

Instead of checking the remote peer address to decide whether we need
media address rewriting, we check our local media address: if it's
local, then we rewrite. This feels awkward, but works and even made
directmedia work properly if you set local_net. (For the record: for
local peers, the SDP media rewrite code is not called, so the
comparison does no harm there.)

ASTERISK-27248 #close

Change-Id: I566be1c33f4d0a689567d451ed46bab9c3861d4f

In 2dee95cc (ASTERISK-27024) and 776ffd77 (ASTERISK-26879) there was
confusion about whether the transport_state->localnet ACL has ALLOW or
DENY semantics.

For the record: the localnet has DENY semantics, meaning that "not in
the list" means ALLOW, and the local nets are in the list.

Therefore, checks like this look wrong, but are right:

    /* See if where we are sending this request is local or not, and if
       not that we can get a Contact URI to modify */
    if (ast_apply_ha(transport_state->localnet, &addr) != AST_SENSE_ALLOW) {
        ast_debug(5, "Request is being sent to local address, "
                     "skipping NAT manipulation\n");

(In the list == localnet == DENY == skip NAT manipulation.)

And conversely, other checks that looked right, were wrong.

This change adds two macro's to reduce the confusion and uses those
instead:

    ast_sip_transport_is_nonlocal(transport_state, addr)
    ast_sip_transport_is_local(transport_state, addr)

ASTERISK-27248 #close

Change-Id: Ie7767519eb5a822c4848e531a53c0fd054fae934

t38_reinvite_response_cb can get called by res_pjsip_session's
session_inv_on_tsx_state_changed in situations where session->channel
is NULL.  If it is, the ast_log warning segfaults because it tries
to get the channel name from a NULL channel.

* Check session->channel and print "unknown channel" when it's NULL.

ASTERISK-27236
Reported by: Ross Beer

Change-Id: I4326e288d36327f6c79ab52226d54905cdc87dc7

The "external_media_address" option on transports is now
resolved using dnsmgr. This allows it to be automatically
refreshed regularly if refreshes are enabled in dnsmgr.
If the system is using a dynamic IP address a dynamic DNS
hostname can be provided to keep the IP address up to
date.

Change-Id: Ia54771720dff0105bde55d5bbb81a3ba437e05b2

BUNDLE is a specification used in WebRTC to allow multiple
streams to use the same underlying transport. This reduces
the number of ICE and DTLS negotiations that has to occur
to 1 normally.

This change implements this by adding support for it to
the RTP SDP module in PJSIP. BUNDLE can be turned on using
the "bundle" option and on an offer we will offer to
bundle streams together. On an answer we will accept any
bundle groups provided. Once accepted each stream is bundled
to another RTP instance for transport.

For the res_rtp_asterisk changes the ability to bundle
an RTP instance to another based on the SSRC received
from the remote side has been added. For outgoing traffic
if an RTP instance is bundled to another we will use the
other RTP instance for any transport related things. For
incoming traffic received from the transport instance we
look up the correct instance based on the SSRC and use it
for any non-transport related data.

ASTERISK-27118

Change-Id: I96c0920b9f9aca7382256484765a239017973c11

arm the t38 webhook always, so we can correctly reject a
T38 negotiation request when t38 is disabled on a channel

Change-Id: Ib1ffe35aee145d4e0fe61dd012580be11aae079d

The stream topology (list of streams and order) is now stored with the
configured PJSIP endpoints and used during the negotiation process.

Media negotiation state information has been changed to be stored
in a separate object. Two of these objects exist at any one time
on a session. The active media state information is what was previously
negotiated and the pending media state information is what the
media state will become if negotiation succeeds. Streams and other
state information is stored in this object using the index (or
position) of each individual stream for easy lookup.

The ability for a media type handler to specify a callback for
writing has been added as well as the ability to add file
descriptors with a callback which is invoked when data is available
to be read on them. This allows media logic to live outside of
the chan_pjsip module.

Direct media has been changed so that only the first audio and
video stream are directly connected. In the future once the RTP
engine glue API has been updated to know about streams each individual
stream can be directly connected as appropriate.

Media negotiation itself will currently answer all the provided streams
on an offer within configured limits and on an offer will use the
topology created as a result of the disallow/allow codec lines.

If a stream has been removed or declined we will now mark it as such
within the resulting SDP.

Applications can now also request that the stream topology change.
If we are told to do so we will limit any provided formats to the ones
configured on the endpoint and send a re-invite with the new topology.

Two new configuration options have also been added to PJSIP endpoints:

max_audio_streams: determines the maximum number of audio streams to
offer/accept from an endpoint. Defaults to 1.

max_video_streams: determines the maximum number of video streams to
offer/accept from an endpoint. Defaults to 1.

ASTERISK-27076

Change-Id: I8afd8dd2eb538806a39b887af0abd046266e14c7

The T38 sdp callback incorrectly has a side effect of incrementing
the media_count.  This can lead to core dumps.

Change-Id: I7bb2f4987de4046ec52cfc34e5ea0662dae32af8

A deadlock can happen between a channel lock and a pjsip session media
container lock.  One thread is processing a reINVITE's SDP and walking
through the session's media container when it waits for the channel lock
to put the determined format capabilities onto the channel.  The other
thread is writing a frame to the channel and processing the T.38 frame
hook.  The T.38 frame hook then waits for the pjsip session's media
container lock.  The two threads are now deadlocked.

* Made the T.38 frame hook release the channel lock before searching the
session's media container.  This fix has been done to several other
frame hooks to fix deadlocks.

ASTERISK-26974 #close

Change-Id: Ie984a76ce00bef6ec9aa239010e51e8dd74c8186

In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE.  This prevents asterisk from exiting
if a module can't be loaded.  If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.

A new API was added to logger: ast_is_logger_initialized().  This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout.  If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.

Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25

If local_net is not defined on a transport, transport_state->localnet
will be NULL. ast_apply_ha will, be default, return AST_SENSE_ALLOW in
this case, causing the external_media_address, if set, to be skipped.

This patch causes us to only check if we are sending within a network if
local_net is defined.

ASTERISK-26879 #close

Change-Id: Ib661c31a954cabc9c99f1f25c9c9a5c5b82cbbfb

The latest Release candidate fails to create RTP streams when IPv6
is not available. Due to the changes made in September the ast_sockaddr
structure passed around to create these streams is always of AF_INET6
type, causing failure when used for IPv4. This patch adds a utility
function to check for availability of IPv6 and applies such check
at startup to determine how to create the ast_sockaddr structures.

ASTERISK-26617 #close

Change-Id: I627a4e91795e821111e1cda523f083a40d0e0c3e

ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.

Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename

This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled.  This variable was only used in lock.c so it
is now initialized in that file only.

ASTERISK-26480 #close

Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966

This change adds support for dual stack automatically. No
configuration is required and the IP address and version
in the SIP messages and SDP will be automatically changed
based on the transport over which the message is being
sent. RTP usage has also been changed to listen on both
IPv4 and IPv6 simultaneously to allow media to flow, and
to allow ICE support on both simultaneously. This also
allows failover between IPv6 and IPv4 to work as expected.

ASTERISK-26309 #close

Change-Id: I235a421d8f9a326606d861b449fa6fe3a030572d

The framehook container could become corrupted if the channel lock is not
held before calling.

Change-Id: I1a6b957a1f7b899eb29a186915f8cccab886a438

Some T.38 implementations may send another re-invite after the initial
one which adds additional negotiation details (such as the max bitrate).
Currently this will fail when passthrough is being done in chan_sip as we
do nothing if T.38 is already active.

Other handlers of T.38 inside of Asterisk (such as res_fax) handle this
scenario so this change adds support for it to chan_sip and res_pjsip_t38.
If a request to negotiate is received while T.38 is already enabled a
new re-INVITE is sent and negotiation is done again.

ASTERISK-26179 #close

Change-Id: I0298494d3da6df3219bbfa4be9aa04015043145c

This backs out item 4 of the 4875e5ac
commit.  Item 4 added the t38_bye_supplement.  Unfortunately, the frame
that it puts into the bridge may or may not be processed by the time the
bridged peer is kicked out of the bridge.  If it is processed then all is
well.  However, if it is not processed then that channel is stuck in fax
mode until it hangs up or maybe if it joins another bridge for T.38
faxing.

ASTERISK-25582

Change-Id: Ib20a03ecadf1bf8a0dcadfadf6c2f2e60919a9f7

Attempting to load a transport from realtime was forcing asterisk into an
infinite recursion loop.  The first thing transport_apply did was to do a
sorcery retrieve by id for an existing transport of the same name. For files,
this just returns the previous object from res_sorcery_config's internal
container, if any.  For realtime, the res_sourcery_realtime driver looks in the
database and finds the existing row but now it has to rehydrate it into a
sorcery object which means calling... transport_apply.  And so it goes.

The main issue with loading from realtime (apart from the loop) was that
transport stores structures and pointers directly in the ast_sip_transport
structure instead of the separate ast_transport_state structure.  This patch
separates those items into the ast_sip_transport_state structure.  The pattern
is roughly the same as res_pjsip_outbound_registration.

Although all current usages of ast_sip_transport and ast_sip_transport_state
were modified to use the new ast_sip_get_transport_state API, the original
items are left in ast_sip_transport and kept updated to maintain ABI
compatability for third-party modules.  They are marked as deprecated and
noted that they're now in ast_sip_transport_state.

ASTERISK-25606 #close
Reported-by: Martin Moučka

Change-Id: Ic7a836ea8e786e8def51fe3f8cce855ea54f5f19

When a channel is in a direct media bridge, a re-INVITE may arrive that forces
Asterisk to re-negotiate the media to a T.38 fax. When this occurs, the bridge
must change its technology to a simple bridge, and re-INVITE the media back
to Asterisk.

Generally, this logic mostly already exists in Asterisk. However, prior to this
patch, there were a few bugs:
(1) The T.38 framehook currently prevents a channel capable of T.38 faxes from
    ever entering into a direct media bridge. This applies even when the only
    media being passed over the channel is audio. This patch fixes this bug
    by having the framehook specify that it defers caring about any frame type.
    This allows the channels to enter into a direct media bridge, which will
    be broken when a re-INVITE is received.
(2) When a re-INVITE is received, nothing instructed the bridging layer to
    re-inspect the allowed bridging technology. This now occurs when either
    a re-INVITE is received from a peer, or when a response is received from
    the far end (that is, when the T.38 state changes to either
    T38_PEER_REINVITE or T38_LOCAL_REINVITE).
(3) chan_pjsip needs to do a small amount of work to prevent a direct media
    bridge from being chosen when a T.38 session is in progress. When a T.38
    session supplement has a t38 datastore - which is added when we detect
    we should start thinking about T.38 on a channel - we now refuse a native
    RTP bridge.
(4) When a BYE request is received, we don't terminate the T.38 session. If
    the other side of a T.38 fax survives the hangup (due to the 'g' flag
    in Dial, for example), we don't currently re-INVITE the media on the
    other channel back to audio. This patch now has res_pjsip_t38 intercept
    BYE requests and inform the far side that the T.38 session is terminated.
    This naturally causes the correct re-INVITEs to be sent.

ASTERISK-25582

Change-Id: Iabd6aa578e633d16e6b9f342091264e4324a79eb

This patch adds some debug statements to res_pjsip_t38. These statements help
to determine which SDP negotiation callbacks are being executed, and, when
a particular callback exits, why a callback may not have applied its logic
to the local or remote SDP.

Change-Id: I61b3fb9183b7ebbb5da8e9f48b59a5d9d7042d77

Modules commonly used the pj_gethostip function for retrieving the
IP address of the host. This function does not cache the result and may
result in a DNS lookup occurring, or additional work. If the DNS
server is unreachable or network issues arise this can cause the
pj_gethostip function to block for a period of time.

This change adds an ast_sip_get_host_ip and ast_sip_get_host_ip_string
function which does the same thing but caches the host IP address at
module load time. This results in no additional work being done each
time the local host IP address is needed.

ASTERISK-25342 #close

Change-Id: I3205deb679b01fa5ac05a94b623bfd620a2abe1e

When a caller calls a FAX number and then hangs up right after the call is
answered then the T.38 re-INVITE automatic reject timer may still be
running after the channel goes away.

* Added session NULL channel checks on the code paths that get executed by
t38_automatic_reject() to prevent a crash when the T.38 re-INVITE
automatic reject timer expires.

ASTERISK-25168
Reported by: Carl Fortin

Change-Id: I07b6cd23815aedce5044f8f32543779e2f7a2403

Calling t38_change_state() sets the t38 state so it makes little sense to
then check the state right after the call for something else.

* Made the code in t38_interpret_parameters() reject or exit T.38 mode as
intended but not implemented.

Change-Id: Ib281263a6ed44da9448132c4e6df1e183b8a3df2

Change-Id: Icf88f9f861c6b2a16e5f626ff25795218a6f2723

When Asterisk originates a channel to an application, the channel is
hung up once the application finishes executing. When the application
in question is SendFax, the Asterisk PJSIP code will attempt to reinvite
the T.38 session to audio after the FAX completes. The hangup of the
channel happens in the midst of this reinvite transaction. In most
circumstances, this works out okay because the BYE is delayed until the
reinvite transaction can complete.

However, if the reinvite that Asterisk sends receives a 401/407
response, then Asterisk's attempt to re-send the reinvite with
authentication will fail. This is because the session supplement in
res_pjsip_t38 makes the assumption that the channel on the session will
always be non-NULL. Since the channel has been hung up, though, the
channel is now NULL. Attempting to operate on the channel causes a
crash.

This patch fixes the issue by ensuring that the channel on the session
is not NULL before attempting to mess with the T.38 framehook.

This patch also contains some corrections for comments that were
incorrect and really confused me when I first started looking at the
code.

ASTERISK-25004 #close
Reported by Mark Michelson

Change-Id: Ic5a1230668369dda4bb13524098aed9306ab45a0