Change-Id: I26d4d2cb9cee924632ff59ef0b30a7e6a1e2b00d

Change-Id: I57081045c72b9fcf12d5c84493278f9272c31b32

A recent change went into Asterisk which added reference counts to the
channels stored in a dial masquerade datastore. Unfortunately this
included a reference to the caller in a dialing operation. While all
of the dialed targets have the datastore removed from them upon dialing
completion this did not occur for the caller, causing it to have a
reference to itself that could go never go away (as it depended on
the destruction of the datastore which only happened when the channel
was destroyed). This resulted in the caller channel remaining on the
system despite it having hung up.

This change does the following to fix this issue:

1. The dial masquerade datastore is now removed from the caller upon
dialing completion, just like the dialed targets.
2. Upon destruction of the caller all the dialed targets are also
removed from the dial masquerade datastore (just in case).
3. The reference to the caller has been removed as it should not be
possible for the datastore to now be valid/useful after the lifetime
of the caller has ended.

ASTERISK-25025 #close

Change-Id: I1ef4ca5ca04980028604cc2af5d2992ac3431b3f

Renamed AST_VECTOR_INSERT to AST_VECTOR_REPLACE because it really
does replace not insert.  The few users of AST_VECTOR_INSERT were
refactored.  Because these are macros, there should be no ABI
compatibility issues.

Added AST_VECTOR_INSERT_AT that actually inserts an element into the
vector at a specific index pushing existing elements to the right.

Added AST_VECTOR_GET_CMP that can retrieve from the vector based
on a user-provided compare function.

Added AST_VECTOR_CALLBACK function that will execute a function
for each element in the vector.  Similar to ao2_callback and
ao2_callback_data functions although the vector callback can take
a variable number of arguments.  This should allow easy migration
to a vector where a container might be too heavy.

Added read/write locked vector and lock manipulation macros.

Added unit tests.

ASTERISK-25045 #close

Change-Id: I2e07ecc709d2f5f91bcab8904e5e9340609b00e0

This adds a test that will fail if any test failed to register. Also fail
if any test registration produced a warning about missing a leading or
trailing slash.

ASTERISK-25053 #close
Reported by: Corey Farrell

Change-Id: I93e50b8fcbcfa7f1f5b41b2c44a51685c09529c3

Apply the negative connection cache setting to all connections,
even those that are not pooled. This ensures that the connection
will not be  re-established before the negative connection cache
time is met.

ASTERISK-22708 #close

Change-Id: I431cc2e8584ab0b6908b3523d0a0e18c9a527271

Format interfaces cannot be unregistered, so the modules that provide them
need to be held open except by shutdown.

ASTERISK-25054 #close
Reported by: Corey Farrell

Change-Id: Iadbd9675bf0d30b8fded5a739b163db3ea2db8f3

The upgrade script for auto DTMF mode (31cd4f4891ec) added in 88b0fa77
failed to add ENUM support for Postgres databases. This requires a
specific import from the sqlalchemy.dialects.postgresql package. This
patch corrects this error, which allows for Postgres update scripts to
be generated.

ASTERISK-24706

Change-Id: I4742ac8efa533cd6f18e0bdd907b339a9aedf015

ASTERISK-25053
Reported by: Corey Farrell

Change-Id: I8c0375dd0818747b2d2e1ceaea87bfbeb2daf8d4

Created autoconf/ast_check_raii.m4: contains AST_CHECK_RAII which
checks compiler requirements for RAII:
gcc: -fnested-functions support
clang: -fblocks (and if required -lBlocksRuntime)
The original check was implemented in configure.ac and now has it's
own file. This function also sets C_COMPILER_FAMILY to either gcc or
clang for use by makefile

Created autoconf/ast_check_strsep_array_bounds.m4 (contains
AST_CHECK_STRSEP_ARRAY_BOUNDS):
which checks if clang is able to handle the optimized strsep & strcmp
functions (linux). If not, the standard libc implementation should be
used instead. Clang + the optimized macro's work with:
strsep(char *, char []), but not with strsepo(char *, char *).
Instead of replacing all the occurences throughout the source code,
not using the optimized macro version seemed easier

See 'define __strcmp_gc(s1, s2, l2) in bits/string2.h':
llvm-comment: Normally, this array-bounds warning are suppressed for
macros, so that unused paths like the one that accesses __s1[3] are
not warned about.  But if you preprocess manually, and feed the
result to another instance of clang, it will warn about all the
possible forks of this particular if statement. Instead of switching
of this optimization, another solution would be to run the preproces-
sing step with -frewrite-includes, which should preserve enough
information so that clang should still be able to suppress the diag-
nostic at the compile step later on.

See also "https://llvm.org/bugs/show_bug.cgi?id=20144"
See also "https://llvm.org/bugs/show_bug.cgi?id=11536"

Makefile.rules: If C_COMPILER_FAMILY=clang then add two warning
suppressions:
-Wno-unused-value
-Wno-parentheses-equality
In an earlier review (reviewboard: 4550 and 4554), they were deemed a
nuisace and less than benefitial.

configure.ac:
Added AST_CHECK_RAII() see earlier
Added AST_CHECK_STRSEP_ARRAY_BOUNDS() see earlier
Removed moved content

ASTERISK-24917
Change-Id: I12ea29d3bda2254ad3908e279b7effbbac6a97cb

This patch adds a new option to cdr.conf, 'newcdrcolumns', that will handle CDR
columns added in Asterisk 1.8. The columns are:
 * peeraccount
 * linkedid
 * sequence
When enabled, the columns in the database entry will be populated with the data
from the CDR.

ASTERISK-24976 #close

Change-Id: I51a57063f4ae5e194a9d933a8df45dc8a4534f0b

A few cases exist where headers of optional_api provders are included but
not needed.  This causes unneeded calls to ast_optional_api_use.

* Don't include optional_api.h from sip_api.h.
* Move 'struct ast_channel_monitor' to channel.h.
* Don't include monitor.h from chan_sip.c, channel.c or features.c.

The move of struct ast_channel_monitor is needed since channel.c depends on
it.  This has no effect on users of monitor.h since channel.h is included
from monitor.h.

ASTERISK-25051 #close
Reported by: Corey Farrell

Change-Id: I53ea65a9fc9693c89f8bcfd6120649bfcfbc3478

Removed the extra space before "MODULEINFO" in res_pjsip_dlg_options.
This extra space prevented any of the dependencies from being seen by
menuselect, so building with default options would fail if PJSIP was
not installed.

This also makes the tool that extracts information for menuselect
tolerant of multiple spaces in the future.

ASTERISK-25033 #close
Reported by: Peter Whisker

Change-Id: Iccd54846f70c4a7a50cb5bf70b7bb5cb4bab3698

When using the force black background command-line option or configuration
option an invalid reset sequence is sent following a coloured output item 
in the CLI, the result is that the colour is not 'turned off' and continues
until the next non-default coloured text output.

A reset sequence is already defined in term.c, but the ast_term_reset
function doesn't use it, instead building it's own invalid sequence and 
returning that.

This patch changes that behaviour, removing the building of a reset sequence
and instead using the pre-built constant 'enddata' which is a suitable reset
sequence for this purpose.

ASTERISK-24896 #close
Reported by: Dan Tucny

Change-Id: I56323899123ae3264900389cae1f5b252aa3bf43

This ensures that refdebug is initialized before AO2_DEBUG if
both are enabled, since AO2_DEBUG allocates a container.

This change also makes AO2_DEBUG initialization critical, a
failure will abort Asterisk startup.  This is needed since
the failure would be caused by reg_containers allocation
failure, and that would result in a segmentation fault by
ao2_container_register later in startup.

ASTERISK-25048 #close
Reported by: Corey Farrell

Change-Id: I9a243ea3fc5653b48b931ba6d61971cb2e530244

The PBX core maintains two hash tables for hints: a container of the
actual hints (hints), along with a container of devices that are watching that
hint (hintdevices). When a dialplan reload occurs, each hint in the hints
container is destroyed; this requires a lookup in the container of devices to
find the device => hint mapping object. In the current code, this performs an
ao2_callback, iterating over each of the device to hint objects in the
hintdevices container. For a large number of hints, this is extremely
expensive: dialplan reloads with 20000 hints could take several minutes
in just this phase.

This patch improves the performance of this step in the dialplan reloads
by caching which devices are watching a hint on the hint object itself.
Since we don't want to create a circular reference, we just cache the
name of the device. This allows us to perform a smarter ao2_callback on
the hintdevices container during hint removal, hashing on the name of the
device and returning an iterator to the matching names. The overall
performance improvement is rather large, taking this step down to a number of
seconds as opposed to minutes.

In addition, this patch also registers the hint containers in the PBX
core with the astobj2 library. This allows for reasonable debugging to
hash collisions in those containers.

ASTERISK-25040 #close
Reported by: Matt Jordan

Change-Id: Iedfc97a69d21070c50fca42275d7b3e714e59360

The sample pjsip.conf has a few comment lines that are missing the
semicolons at the start of the comment, causing the config to fail
load.

Change-Id: I776a38c916a7df7ee3e072fd0b21dbf4cc457352

Scenario:
Alice calls Bob. Bob performs a blond transfer to Carol. Carol rejects
the incoming call (or some other immediate circumstance causes Carol not
to answer the call)

What occurs in this case is that when the bridge between Alice and Bob
breaks, Alice is told to masquerade into Bob's channel that had placed
the call to Carol. The actual masquerade goes down without a hitch.
However, a channel fixup callback that attempts to publish dial events
over Stasis has a crash. The reason for this crash is that the datastore
on Bob's channel that placed the outbound call to Carol only had a bare
pointer to Carol's channel. Since Carol rejected the incoming call,
Carol's channel has been hung up and freed, meaning accessing her
channel results in a crash.

The fix here is simple. The dial fixup code has been altered to hold
references to the involved channels and to drop those references when
freeing data.

ASTERISK-25025 #close
Reported by Chet Stevens

Change-Id: I54eedda207b8ec7a69263353b43abe5746aea197

The build system now scans additional sources when generating
moduleinfo for menuselect.  Unfortunately the extra sources
for format_mp3 only exist if downloaded.

Use the Makefile macro 'wildcard' to allow moduleinfo generator
to ignore sources that do not exist.

Change-Id: I596604713b7345ce994f32197f8f6bfd9bcf4170

The res_ari_device_states module depends on res_stasis_device_state,
not res_stasis_device_states.

Change-Id: I26e02ad37f9e36bcc859867e2fad1b90452ec3de

This patch has two main purposes:

1) Improve warning messages when ACLs are configured improperly.
2) Prevent misconfigured ACLs from allowing potentially unwanted
traffic.

To acomplish point (2) in most cases, whatever configuration object that
the ACL belonged to was not allowed to load.

The one exception is res_pjsip_acl. In that case, ACLs are their own
configuration object. Furthermore, the module loading code has no
indication that a ACL configuration had a failure. So the tactic taken
here is to create an ACL that just blocks everything.

ASTERISK-24969
Reported by Corey Farrell

Change-Id: I2ebcb6959cefad03cea4d81401be946203fcacae

Some telco switches occasionally ignore ISDN RESTART requests.  The fix
for ASTERISK-19608 added an escape clause for B channels in the restarting
state if the telco ignores a RESTART request.  If the telco fails to
acknowledge the RESTART then Asterisk will assume the telco acknowledged
the RESTART on the second call attempt requesting the B channel by the
telco.  The escape clause is good for dealing with RESTART requests in
general but it does cause the next call for the restarting B channel to be
rejected if the telco insists the call must go on that B channel.

chan_dahdi doesn't really need to issue a RESTART request in response to
receiving a cause 44 (Requested channel not available) code.  Sending the
RESTART in such a situation is not required (nor prohibited) by the
standards.  I think chan_dahdi does this for historical reasons to deal
with buggy peers to get channels unstuck in a similar fashion as the
chan_dahdi.conf resetinterval option.

* Add the chan_dahdi.conf force_restart_unavailable_chans compatability
option that when disabled will prevent chan_dahdi from trying to RESTART
the channel in response to a cause 44 code.

ASTERISK-25034 #close
Reported by: Richard Mudgett

Change-Id: Ib8b17a438799920f4a2038826ff99a1884042f65

When the PJSIP pjsip_regc_send function is invoked and an error
status returned the caller currently decrements the reference count
of the client state that it just incremented, assuming the
registration callback would not have been invoked. In practice
this is not correct. If the failure happens after the transaction
has been set up the callback will still be invoked. This will
cause the reference count to be incorrectly decremented twice, once
by the registration callback and second by the caller of
pjsip_regc_send.

This change makes it so that whether the callback is invoked or
not is known by the caller of pjsip_regc_send. Depending on
this it can know whether it is responsible for decrementing the
reference count of the client state or not.

ASTERISK-25037 #close
Reported by: Joshua Colp

Change-Id: I749dc12f3a22115c49c5d7d95ff42a5fa45319de