Previous commits maintained compatibility with older remote console
clients as well as maintaining all API's.

Remove the following compatibility code:
* ast_cli_generatornummatches.
* Remote command "_command nummatches".
* Sorting / duplicate removal by remote console.

Change-Id: I59e6ce94fa57ae564888442049695f7e46746437

Change-Id: Ib81318f4ee52a5e73b003316e13fe9be1dd897a1

* Stop using "_COMMAND NUMMATCHES" on remote consoles.  Using this
  command had doubled the amount of work needed from the Asterisk
  daemon for each completion request.
* Fix code formatting.
* Remove static buffer used to send the command, use the same buffer
  that will receive the results.
* Move sort from ast_cli_display_match_list.

Change-Id: Ie2211b519a3d4bec45bf46e0095bdd01d384cb69

This rewrites ast_el_strtoarr to use vector's internally, but still
return the original NULL terminated array of strings.

Change-Id: Ibfe776cbe14f750effa9ca360930acaccc02e957

* Stop estimating line count, just print until we run out of matches.
* Stop freeing entries, the caller does that anyways.
* Stop calculating / returning numoutput, it was ignored.

Change-Id: I7f92afa8bea92241a95227587367424c8c32a5cb

Reduce the signal monitoring thread file descriptor use from two to one
on systems that support eventfd.

Change-Id: Id4041a237d481ff699639e153ea6982fee14a462

The remote console socket path is the combination of asterisk.conf
settings astrundir from [directories] and astctl from [files].
Unconditionally combine the two strings after processing all values
to ensure we end up with the correct socket path.

ASTERISK-27415

Change-Id: Ib1e2805d55d6b0955c6430a1a2a93acbf9b091e8

The media frame cache gets in the way of finding use after free errors of
media frames.  Tools like valgrind and MALLOC_DEBUG don't know when a
frame is released because it gets put into the cache instead of being
freed.

* Added the "cache_media_frames" option to asterisk.conf.  Disabling the
option helps track down media frame mismanagement when using valgrind or
MALLOC_DEBUG.  The cache gets in the way of determining if the frame is
used after free and who freed it.  NOTE: This option has no effect when
Asterisk is compiled with the LOW_MEMORY compile time option enabled
because the cache code does not exist.

To disable the media frame cache simply disable the cache_media_frames
option in asterisk.conf and restart Asterisk.

Sample asterisk.conf setting:
[options]
cache_media_frames=no

ASTERISK-27413

Change-Id: I0ab2ce0f4547cccf2eb214901835c2d951b78c00

handle_quit has been disabled since 2003, remove it.

Change-Id: Idc3aaa6c81676160547078f9b71e8aa43de2db18

An admin can configure app_minivm with an externnotify program to be run
when a voicemail is received.  The app_minivm application MinivmNotify
uses ast_safe_system() for this purpose which is vulnerable to command
injection since the Caller-ID name and number values given to externnotify
can come from an external untrusted source.

* Add ast_safe_execvp() function.  This gives modules the ability to run
external commands with greater safety compared to ast_safe_system().
Specifically when some parameters are filled by untrusted sources the new
function does not allow malicious input to break argument encoding.  This
may be of particular concern where CALLERID(name) or CALLERID(num) may be
used as a parameter to a script run by ast_safe_system() which could
potentially allow arbitrary command execution.

* Changed app_minivm.c:run_externnotify() to use the new ast_safe_execvp()
instead of ast_safe_system() to avoid command injection.

* Document code injection potential from untrusted data sources for other
shell commands that are under user control.

ASTERISK-27103

Change-Id: I7552472247a84cde24e1358aaf64af160107aef1

GCC 7 has added capability to produce warnings, this fixes most of those
warnings.  The specific warnings are disabled in a few places:

* app_voicemail.c: truncation of paths more than 4096 chars in many places.
* chan_mgcp.c: callid truncated to 80 chars.
* cdr.c: two userfields are combined to cdr copy, fix would break ABI.
* tcptls.c: ignore use of deprecated method SSLv3_client_method().

ASTERISK-27156 #close

Change-Id: I65f280e7d3cfad279d16f41823a4d6fddcbc4c88

Setting maxfiles (maximum number of open files) has no practical
effect on a remote asterisk (rasterisk, rasterisk -x).

It has an ill effect of printing an extra message, which
may be annoying in case of -x.

ASTERISK-27105 #close

Change-Id: Iaf9eb344e4b4b517df91b736b27ec55f6a6921a2

This API was not actively maintained, was not added to new modules
(such as res_pjsip), and there exist better alternatives to acquire the
same information, such as the ARI.

Change-Id: I4b2185a83aeb74798b4ad43ff8f89f971096aa83

This change adds support for socket activation of certain SOCK_STREAM
listeners in Asterisk:
* AMI / AMI over TLS
* CLI
* HTTP / HTTPS

Example systemd units are provided.  This support extends to any socket
which is initialized using ast_tcptls_server_start, so any unknown
modules using this function will support socket activation.

Asterisk continues to function as normal if socket activation is not
enabled or if systemd development headers are not available during
build.

ASTERISK-27063 #close

Change-Id: Id814ee6a892f4b80d018365c8ad8d89063474f4d

In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE.  This prevents asterisk from exiting
if a module can't be loaded.  If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.

A new API was added to logger: ast_is_logger_initialized().  This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout.  If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.

Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25

Dynamic payload types were statically defined in Asterisk. This unfortunately
limited the number of dynamic payloads that could be registered. With this patch
dynamic payload type numbers are now assigned dynamically and per RTP instance.
However, in order to limit any issues where some clients expect the old
statically defined value this patch makes it so the value Asterisk used to pre-
designate is used for the dynamic assignment if available.

An option, "rtp_use_dynamic", has also been added (can be set in asterisk.conf)
that turns the new dynamic behavior on or off. When off it reverts back to using
statically defined payload values. This option defaults to "yes" in Asterisk 15.

ASTERISK-26515 #close
patches:
  ASTERISK-26515.diff submitted by jcolp (license 5000

Change-Id: I7653465c5ebeaf968f1a1cc8f3f4f5c4321da7fc

The mechanism used for detecting the maximum log level compiled into the
linked pjproject did not work.  The API call simply stores the requested
level into an integer and does no range checking.  Asterisk was assuming
that there was range checking and limited the new value to the allowable
range.  To get the actual maximum log level compiled into the linked
pjproject we need to get and save off the initial set log level from
pjproject.  This is the maximum log level supported.

* Get and save off the initial log level setting before altering it to the
desired level on startup.  This has to be done by a macro rather than
calling a core function to avoid incorrectly linking pjproject.

* Split the initial log level warning messages to warn if the linked
pjproject cannot support the requested startup level and if it is too low
to get the pjproject buildopts for "pjproject show buildopts".

* Adjust the CLI "pjproject set log level" to check the saved max log
level and to generate normal output messages instead of a warning message.

ASTERISK-26743 #close

Change-Id: I40aa76653e2a1dece66c3f8734594b4f0471cfb4

Use of the new logging is as simple as issuing the new CLI command or
setting the new pjproject.conf option.

Other options that can affect the logging are how you have the pjproject
log levels mapped to Asterisk log types in pjproject.conf and if you have
configured Asterisk to log the DEBUG type messages.  Altering the
pjproject.conf level mapping shouldn't be necessary for most installations
as the default mapping is sensible.  Configuring Asterisk to log the DEBUG
message type is standard practice for collecting debug information.

* Added CLI "pjproject set log level" command to dynamically adjust the
maximum pjproject log message level.

* Added CLI "pjproject show log level" command to see the currently set
maximum pjproject log message level.

* Added pjproject.conf startup section "log_level" option to set the
initial maximum pjproject log message level so all messages could be
captured from initialization.

* Set PJ_LOG_MAX_LEVEL to 6 to compile in all defined logging levels into
bundled pjproject.  Pjproject will use the currently set run time log
level to determine if a log message is generated just like Asterisk
verbose and debug logging levels.

* In log_forwarder(), made always log enabled and mapped pjproject log
messages.  DEBUG mapped log messages are no longer gated by the current
Asterisk debug logging level.

* Removed RAII_VAR() from res_pjproject.c:get_log_level().

ASTERISK-26630 #close

Change-Id: I6dca12979f482ffb0450aaf58db0fe0f6d2e5389

If a TCP/TLS connection was pending (not accepted and not timed out) during
unload of chan_sip, Asterisk would segfault when trying to send a signal to
a thread whose thread ID hadn't been recorded yet. This commit fixes that by
recording the thread ID before calling the blocking connect() syscall.
This was a regression introduced by 776a1438.

The above wasn't enough to fix the segfault, which was now delayed to the
point where connect() timed out. Therefore, it was necessary to also remove
the SA_RESTART flag from the SIGURG sigaction so that pthread_kill() could be
used to interruput the connect() syscall.
This was a regression introduced by 5d313f51.

ASTERISK-26586 #close

Change-Id: I76fd9d47d56e4264e2629bce8ec15fecba673e7b

Libedit 3.1 is not build with unicode on as a default and so the
prototype for the el_gets callback changed from expecting a char buffer
to accepting a wchar buffer.  If ast_el_read_char isn't changed,
the cli reads garbage from teh terminal.

Added a configure test for (*el_rfunc_t)(EditLine *, wchar_t *) and
updated ast_el_read_char to use the HAVE_ define to detemrine whether
to use char or wchar.

ASTERISK-26592 #close

Change-Id: I9099b46f68e06d0202ff80e53022a2b68b08871a

Since adding all remaining rates of Signed Linear (ASTERISK-24274), SILK
(Gerrit 3136) and Codec 2 (ASTERISK-26217), no RTP Payload Type is left in the
dynamic range (96-127). RFC 3551 section 3 allows to reassign other ranges.
Consequently, when the dynamic range is exhausted, this change utilizes payload
types in the range between 35 and 63 giving room for another 29 payload types.

ASTERISK-26311 #close

Change-Id: I7bc96ab764bc30098a178b841cbf7146f9d64964

It is only safe to run ast_register_cleanup callbacks when all modules
have been unloaded.  Previously these callbacks were run during graceful
shutdown, making it possible to crash during shutdown.

ASTERISK-26513 #close

Change-Id: Ibfa635bb688d1227ec54aa211d90d6bd45052e21

ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.

Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename

This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled.  This variable was only used in lock.c so it
is now initialized in that file only.

ASTERISK-26480 #close

Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966

This allows asterisk to compiled with LOW_MEMORY to load modules built
without LOW_MEMORY.

ASTERISK-26398 #close

Change-Id: I24b78ac9493ab933b11087a8b6794f3c96d4872d

Move the function outside the conditional block that excludes
LOW_MEMORY.

ASTERISK-26273 #close

Change-Id: Ic290fa128222c410c3531107e30efacabc8493b4

Previous versions of Asterisk did not require verbose to be specified in
logger.conf for the console channel, if it was requested by command line
or asterisk.conf it just worked.  This change causes Asterisk to always
enable verbose in the console channel level mask.  Verbose is displayed
on consoles if requested by command line, option_verbose or 'core set
verbose'.

This also delays initialization of the logger until after threadstorage
is initialized.  Initializing too early can cause messages to be printed
multiple times to the console (stdout).

ASTERISK-26391 #close

Change-Id: I52187d67c2fcb3efd5561bf04b3e5e23e5ee8a04

Without this change, a 'core restart' would kill the astcanary forever
if you're not running as root. Both with and without this patch, the
scheduling priority was still SCHED_RR after restart.

Additionally, the astcanary is now spawned if you start with high
priority and Asterisk doesn't get a chance to lower it. For example
through: `chrt -r 10 sudo -u asterisk asterisk -c`

Also reap killed astcanary processes on core restart.

ASTERISK-26352 #close

Change-Id: Iacb49f26491a0717084ad46ed96b0bea5f627a55

Previously only the canary checking thread itself had its priority set
to SCHED_OTHER. Now all threads are traversed and adjusted.

ASTERISK-19867 #close
Reported by: Xavier Hienne

Change-Id: Ie0dd02a3ec42f66a78303e9c1aac28f7ed9aae39

sd_notify() is used to notify systemd of changes to the status of the
process. This allows the systemd daemon to know when the process
finished loading (and thus only start another program after Asterisk has
finished loading).

To use this, use a systemd unit with 'Type=notify' for Asterisk.

This commit also adds the function ast_sd_notify(), a wrapper around
sd_notify that does nothing if not built with systemd support.

Also adds support for libsystemd detection in the configure script.

Change-Id: Ied6a59dafd5ef331c5c7ae8f3ccd2dfc94be7811

If sysinfo() is available, but not sysctl() or swapctl() the
printing code for swap buffer sizes is incorrectly omitted.
The above condition happens with musl c-library.

Fix #if rule to consider defined(HAVE_SYSINFO). And also
remove the redundant || defined(HAVE_SYSCTL) which was
incorrectly there to start with. Now swap information is
displayed only if an actual libc function to get it is
available.

This also fixes warnings previously seen with musl libc:

   [CC] asterisk.c -> asterisk.o
asterisk.c: In function 'handle_show_sysinfo':
asterisk.c:773:6: warning: variable 'totalswap' set but not used
 [-Wunused-but-set-variable]
  int totalswap = 0;
      ^~~~~~~~~
asterisk.c:770:11: warning: variable 'freeswap' set but not used
 [-Wunused-but-set-variable]
  uint64_t freeswap = 0;
           ^~~~~~~~

Change-Id: I1fb21dad8f27e416c60f138c6f2bff03fb626eca

The Exchanging Device and Mailbox States could not working
if the Entity ID (EID) is not set manually and can't be obtained
from ethernet interface.

This patch replaces debug message to warning
and addes missing description about option 'entityid' to
asterisk.conf.sample.

With this patch the asterisk also:
(1) decline loading the modules which won't work without EID:
    res_corosync and res_pjsip_publish_asterisk.
(2) warn if EID is empty on loading next modules:
    pbx_dundi, res_xmpp

Starting with v197 systemd/udev will automatically assign "predictable"
names for all local Ethernet interfaces.
This patch also addes some new ethernet prefixes "eno" and "ens".

ASTERISK-26164 #close

Change-Id: I72d712f1ad5b6f64571bb179c5cb12461e7c58c6

Errors during startup result in an exit.  These error branches should be
calling ast_run_atexit(0) to ensure mandatory cleanup is run.

ASTERISK-26267 #close

Change-Id: If226f2326ae2df7add20040696132214cf2bb680

This ensures startup is canceled due to allocation failures from the
following initializations.
* channel.c: ast_channels_init
* config_options.c: aco_init

ASTERISK-26265 #close

Change-Id: I911ed08fa2a3be35de55903e0225957bcdbe9611

Upcoming features will require the generation and persistence
of a UUID.

Change-Id: I3ec0062427e133217db6ef496a4216f427c3b92d

With CLI "core show settings", simply the parameter maxfiles of the file
asterisk.conf was shown. If that parameter was not set, nothing was displayed
although the environment might have set a default number itself. Or if maxfiles
were not granted (completely), still maxfiles was shown. Now, the maximum number
of possible file descriptors in the environment is shown.

ASTERISK-26097

Change-Id: I2df5c58863b5007b34b77adbe28b885dfcdf7e0b

Add Uptime and LastReload to event FullyBooted.

ASTERISK-26058 #close
Reported by: Niklas Larsson

Change-Id: I909b330801c0990d78df9b272ab0adc95aecb15e

When 2d7a4a33 was merged, it missed the fact that Verbose log messages
are formatted and handled by 'verbosers'. Verbosers are registered
functions that handle verbose messages only; they exist as a separate
class of callbacks. This was done to handle the 'magic' that must be
inserted into Verbose messages sent to remote consoles, so that the
consoles can format the messages correctly, i.e., the leading
tabs/characters.

In reality, verbosers are a weird appendage: they're a separate class of
formatters/message handlers outside of what handles all other log
messages in Asterisk. After some code inspection, it became clear that
simply passing a Verbose message along with its 'sublevel' importance
through the normal logging mechanisms removes the need for verbosers
altogether.

This patch removes the verbosers, and makes the default log formatter
aware that, if the log channel is a console log, it should simply insert
the 'verbose magic' into the log messages itself. This allows the
console handlers to interpret and format the verbose message
themselves.

This simplifies the code quite a lot, and should improve the performance
of printing verbose messages by a reasonable factor:
(1) It removes a number of memory allocations that were done on each
    verobse message
(2) It removes the need to strip the verbose magic out of the verbose
    log messages before passing them to non-console log channels
(3) It now performs fewer iterations over lists when handling verbose
    messages

Since verbose messages are now handled like other log messages (for the
most part), the JSON formatting of the messages works as well.

ASTERISK-25425

Change-Id: I21bf23f0a1e489b5102f8a035fe8871552ce4f96

Locking some objects like sorcery objects can be tricky because the underlying
ao2 object may not be the same for all callers.  For instance, two threads that
call ast_sorcery_retrieve_by_id on the same aor name might actually get 2
different ao2 objects if the underlying wizard had to rehydrate the aor from a
database. Locking one ao2 object doesn't have any effect on the other even if
those objects had locks in the first place.

Named locks allow access control by keyspace and key strings.  Now an "aor"
named "1000" can be locked and any other thread attempting to lock "aor" "1000"
will wait regardless of whether the underlying ao2 object is the same or not.
Mutex and rwlocks are supported.

This capability will initially be used to lock an aor when multiple threads may
be attempting to prune expired contacts from it.

Change-Id: If258c0b7f92b02d07243ce70e535821a1ea7fb45

During stress testing, we have frequently seen crashes occur because a
CLI or AMI command attempts to access information that is in the process
of being destroyed.

When addressing how to fix this issue, we initially considered fixing
individual crashes we observed. However, the changes required to fix
those problems would introduce considerable overhead to the nominal
case. This is not reasonable in order to prevent a crash from occurring
while Asterisk is already shutting down.

Instead, this change makes it so AMI and CLI commands cannot be executed
if Asterisk is being shut down. For AMI, this is absolute. For CLI,
though, certain commands can be registered so that they may be run
during Asterisk shutdown.

ASTERISK-25825 #close

Change-Id: I8887e215ac352fadf7f4c1e082da9089b1421990

Background here:
http://lists.digium.com/pipermail/asterisk-dev/2016-January/075266.html

From CHANGES:
 * To help insure that Asterisk is compiled and run with the same known
   version of pjproject, a new option (--with-pjproject-bundled) has been
   added to ./configure.  When specified, the version of pjproject specified
   in third-party/versions.mak will be downloaded and configured.  When you
   make Asterisk, the build process will also automatically build pjproject
   and Asterisk will be statically linked to it.  Once a particular version
   of pjproject is configured and built, it won't be configured or built
   again unless you run a 'make distclean'.

   To facilitate testing, when 'make install' is run, the pjsua and pjsystest
   utilities and the pjproject python bindings will be installed in
   ASTDATADIR/third-party/pjproject.

   The default behavior remains building with the shared pjproject
   installation, if any.

Building:

   All you have to do is include the --with-pjproject-bundled option on
   the ./configure command line (and remove any existing --with-pjproject
   option if specified).  Everything else is automatic.

Behind the scenes:

   The top-level Makefile was modified to include 'third-party' in the
   list of MOD_SUBDIRS.

   The third-party directory was created to contain any third party
   packages that may be needed in the future.  Its Makefile automatically
   iterates over any subdirectories passing on targets.

   The third-party/pjproject directory was created to house the pjproject
   source distribution.  Its Makefile contains targets to download, patch
   configure, generate dependencies, compile libs, apps and python bindings,
   sanitized build.mak and generate a symbols list.

   When bootstrap.sh is run, it automatically includes the configure.m4
   file in third-party/pjproject.  This file has a macro to download and
   conifgure pjproject and get and set PJPROJECT_INCLUDE, PJPROJECT_DIR
   and PJPROJECT_BUNDLED.  It also tests for the capabilities like
   PJ_TRANSACTION_GRP_LOCK by parsing preprocessor output as opposed to
   trying to compile.  Of course, bootstrap.sh is only run once and the
   configure file is incldued in the patch.

   When configure is run with the new options, the macro in configure.m4
   triggers the download, patch, conifgure and tests.  No compilation is
   performed at this time.  The downloaded tarball is cached in /tmp so
   it doesn't get downloaded again on a distclean.

   When make is run in the top-level Asterisk source directory, it will
   automatically descend all the subdirectories in third_party just as it
   does for addons, apps, etc.  The top-level Makefile makes sure that
   the 'third-party' is built before 'main' so that dependencies from the
   other directories are built first.

   When main does build, a new shared library (libasteriskpj) is created that
   links statically to the pjproject .a files and exports all their symbols.
   The asterisk binary links to that, just as it does with libasteriskssl.

   When Asterisk is installed, the pjsua and pjsystest apps, and the pjproject
   python bindings are installed in ASTDATADIR/third-party/pjproject.  This
   will facilitate testing, including running the testsuite which will be
   updated to check that directory for the pjsua module ahead of the system
   python library.

Modules should continue to depend on pjproject if they use pjproject APIs
directly.  They should not care about the implementation.  No changes to any
res_pjsip modules were made.

Change-Id: Ia7a60c28c2e9ba9537c5570f933c1ebcb20a3103