There are times when you need to troubleshoot issues with bundled
pjproject or add new features that need to be pushed upstream
but...

* The source directory created by extracting the pjproject tarball
  is not scanned for code changes so you have to keep forcing
  rebuilds.
* The source directory isn't a git repo so you can't easily create
  patches, do git bisects, etc.
* Accidentally doing a make distclean will ruin your day by wiping
  out the source directory, and your changes.
* etc.

This commit makes that easier.
See third-party/pjproject/README-hacking.md for the details.

ASTERISK-29824

Change-Id: Idb1251040affdab31d27cd272dda68676da9b268

OpenSSL is one of those packages that often have alternatives
with later versions.  For instance, CentOS/EL 7 has an
openssl package at version 1.0.2 but there's an openssl11
package from the epel repository that has 1.1.1.  This gets
installed to /usr/include/openssl11 and /usr/lib64/openssl11.
Unfortunately, the existing --with-ssl and --with-crypto
./configure options expect to point to a source tree and
don't work in this situation.  Also unfortunately, the
checks in ./configure don't use pkg-config.

In order to make this work with the existing situation, you'd
have to run...
./configure --with-ssl=/usr/lib64/openssl11 \
    --with-crypto=/usr/lib64/openssl11 \
    CFLAGS=-I/usr/include/openssl11

BUT...  those options don't get passed down to bundled pjproject
so when you run make, you have to include the CFLAGS again
which is a big pain.

Oh...  To make matters worse, although you can specify
PJPROJECT_CONFIGURE_OPTS on the ./configure command line,
they don't get saved so if you do a make clean, which will
force a re-configure of bundled pjproject, those options
don't get used.

So...

* In configure.ac... Since pkg-config is installed by install_prereq
  anyway, we now use it to check for the system openssl >= 1.1.0.
  If that works, great.  If not, we check for the openssl11
  package. If that works, great.  If not, we fall back to just
  checking for any openssl.  If pkg-config isn't installed for some
  reason, or --with-ssl=<dir> or --with-crypto=<dir> were specified
  on the ./configure command line, we fall back to the existing
  logic that uses AST_EXT_LIB_CHECK().

* The whole OpenSSL check process has been moved up before
  THIRD_PARTY_CONFIGURE(), which does the initial pjproject
  bundled configure, is run.  This way the results of the above
  checks, which may result in new include or library directories,
  is included.

* Although not strictly needed for openssl, We now save the value of
  PJPROJECT_CONFIGURE_OPTS in the makeopts file so it can be used
  again if a re-configure is triggered.

ASTERISK-29693

Change-Id: I341ab7603e6b156aa15a66f43675ac5029d5fbde

As described in the issue, /tmp is not a suitable location for a
large amount of cached media files, since most distributions make
/tmp a RAM-based tmpfs mount with limited capacity.

I opted for a location that can be configured separately, as opposed
to using a subdirectory of spooldir, given the different storage
profile (transient files vs files that might stay there indefinitely).

This commit just makes the cache directory configurable, but leaves
it at /tmp by default, to ensure backwards compatibility.

A future commit that only targets master could change the default
location to something more sensible such as /var/tmp/asterisk. At
that point, the cachedir could be created and cleaned up during
uninstall by the Makefile script.

ASTERISK-29143

Change-Id: Ic54e95199405abacd9e509cef5f08fa14c510b5d

Furthermore, the nowhere used compress is removed.

ASTERISK-28816

Change-Id: I77daab80cfabb56d51c3ea6b1d14bd9b9fbc577c

When building check_expr2 with ASLR PIE hardening enabled the linker
fails. This is resolved by adding the regular compiler flags when
building the object files from ast_expr2f.c and ast_expr2.c.

Note: The STANDALONE define is removed because it is already defined in
_ASTCFLAGS. YY_NO_INPUT is defined so that the compile survives
'--enable-dev-mode'.

Also, a Makefile variable "CROSS_COMPILING" is added so that the
build system doesn't try to run check_expr2 when cross-compiling,
because that will fail the build as will.

ASTERISK-28685 #close

Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Change-Id: If435b7db9f9ad8266245bda51c81c220f9658915

Change-Id: Ib3111b151d37cbda40768cf2a8a9c6cf6c5c7cbd

This fixes build warnings found by GCC 8.  In some cases format
truncation is intentional so the warning is just suppressed.

ASTERISK-27824 #close

Change-Id: I724f146cbddba8b86619d4c4a9931ee877995c84

Change-Id: I617a96ebb83ec99f5d3176bbbee2d2a272ccb203

ASTERISK-27761

Change-Id: Ib17a7415297a210cfcdbf149e4df9b6edadbfab6

ASTERISK-27670

Change-Id: I07a8ef8bbd6001e25711fa1bff152eb6c9efa729

This removes the embedded copy of editline from the Asterisk source
tree, making a system copy of libedit mandatory in Asterisk 16+.

ASTERISK-27634 #close

Change-Id: Iedb64ad92acb78419f3caefedaa2bb7cd2a1a33f

OSX does not support 'readlink -f' or 'sed -r'.  Replace readlink with
the GNU make macro 'realpath'.  Replace sed with grep in one place, cut
in the other.

ASTERISK-27332

Change-Id: I5d34ecca905384decb22ead45c913ae5e8aff748

menuselect detects compiler support for multiple styles of weak
functions.  This is a remnant from 2013 when OPTIONAL_API required weak
functions.  It is no longer correct for menuselect to switch
dependencies from optional to required based on lack of weak function
support.

Note an issue remains - dependencies should switch from optional to
required based on OPTIONAL_API being enabled or disabled.  I don't think
this is possible.  menuselect needs to know at startup if OPTIONAL_API
is enabled or disabled, so the only way to fix this is to remove
OPTIONAL_API from menuselect and create a configure option.  I've left
the code that switches in place but it's preprocessed out.

Additionally removed:
- WEAKREF variable from Asterisk makeopts.in.
- Related disabled code from test_utils.
- Pointless AC_REVISION call from menuselect/configure.ac.

Change-Id: Ifa702e5f98eb45f338b2f131a93354632a8fb389

This check is being added to make it easier for end-users of third party
open source Opus modules.  This was removed by ASTERISK-26426 but only
the module needed to be removed.

Change-Id: I62b9cd0c4fa8a77596ab0e042948a643a1152677

The configure option to disable XML documentation does not currently
work.  This patch makes it effective, but also causes an ABI change by
removing the ast_xmldoc_* symbols.  Disabling xmldoc also prevents docs
from being automatically generated, but they can still be manually
generated with 'make doc/core-en_US.xml'.

ASTERISK-26639

Change-Id: Ifac562340c09f80c83e0203de098fcac93bf8c44

Beanstalkd is a simple to use job queue. It provides a means to
create multiple job queues called "tubes". Each tube can store
multiple jobs, with varying priorities with the queue. Queue
processing is available via a simple TCP socket or via well defined
libraries, avaialble at
https://github.com/kr/beanstalkd/wiki/client-libraries

This module is based upon the beanstalk-client library, available
for download at: https://github.com/deepfryed/beanstalk-client

Change-Id: I5fe4089a34ab3b39230786d9bbfddafa56715f48

'--with-pjproject-bundled' is now the default when running
./configure. It can be disabled with '--without-pjproject-bundled'.

To make building without an internet connection easier, a new
./configure option '--with-download-cache' was added that sets
the cache for externals (like pjproject, the codecs and the DPMA),
AND the sounds files.  It can also be specified as an environment
variable named "AST_DOWNLOAD_CACHE".  The existing
'--with-sounds-cache' option / SOUNDS_CACHE_DIR env variable and
'--with-externals-cache' option / EXTERNALS_CACHE_DIR env variable
remain and if specified, will override '--with-downloads-cache'.

ASTERISK-27189

Change-Id: Ifa9783fddf44aafadb060c9feba713dfa81d38ce

Use -Wno-format-truncation only if supported by compiler.

ASTERISK-27171 #close

Change-Id: Iac0aed7a5bcaa16c21b7d62c4e4678d244c4ccb6

Some systems (like macOS) require BIND_8_COMPAT to be defined so that
the nameser libraries are, well, BIND8 compatible.

Change-Id: If79fc27a64f90de1835b5aa3aadfa9be22bd16b0

This has not worked for some time and is no longer actively maintained.

Change-Id: I5110b0db69c152761b58fa025cb0a53b0e544d99

After configuring Asterisk with '--with-pjproject-bundled' the configure/build
process attempts to download pjproject from its download site. Currently, a
timeout of 10 seconds is used that will stop the download process if pjproject
has not been fully downloaded in that time. For some systems this was not enough
time and the process was timing out too early.

This patch raises the download timeout value to '60'. Also, this patch fixes
another bug where the DOWNLOAD_TIMEOUT variable was not being properly exported
due to a naming error. DOWNLOAD_MAX_TIMEOUT is now properly renamed to
DOWNLOAD_TIMEOUT.

ASTERISK-26814 #close

Change-Id: Ia56e4e8a3d39db76bc8a1852b2cf07ec10b39842

Adds the import tool for converting a HRIR database to hrirs.h

ASTERISK-26292

Change-Id: I51eb31b54c23ffd9b544bdc6a09d20c112c8a547

The download process now has a timeout which will cause wget to retry
if it stops retrieving data for 5 seconds and fetch and curl to timeout
if the whole retrieval take smore than 30 seconds.

If the tarball retrieval works, the MD5SUM file is retrieved from
the downloads site and the md5 checksum is verified.

If either the tarball retrieval or MD5SUM retrieval fails, or the
checksums don't match, the entire process is retried once.  If it
fails again, any incomplete tarball is deleted.

.DELETE_ON_ERROR: was also added to the Makefile.  Not only does
this delete the tarball on failure, it till also delete corrupted
library files from the pjproject source directory should they
fail to build correctly.

Tested all the way back to FreeBSD 9, CentOS 6, Debian 6 and
Ubuntu 14.

Change-Id: Iea7d33b96a31622ab1b6e54baebaf271959514e1

Adds libfftw3 to the build chain that is is going to be used for binaural
synthesis by bridge_softmix.

ASTERISK-26292

Change-Id: Iedc2f174e4ccb39ae5d9e698e339c6a17155867b

Added tests for bzip2, tar, patch, sed and nm to configure.ac.

Set DOWNLOAD_TO_STDOUT to a working command line regardless of
whether the download program is wget, curl or fetch.

Added a 'configure.m4' file to the third-party directory which takes
care of calling any third-party project setup.  Had to move some
pjproject_bundled stuff up in configure.ac so it was called before
the third-party configure macro.

The pjproject tarball is now downloaded to the externals_cache_dir if
it was specified on the ./configure command line

Removed regeneration of the pjproject aconfigure file.  It was only
needed for an old patch that no longer applies.

Converted the tests for symbols to explicit tests since we know that
they're now available in the bundled version.  Saves a little time
during configure.

ASTERISK-26416 #close
Reported-by: Corey Farrell

Change-Id: Id1d94251c0155f8dd41b7de7067f35cfbaafbb9b
(cherry picked from commit e6b0053d)
(cherry picked from commit a0d02f38)

This reverts commit 40aa2813.

ASTERISK-26426 #close

Change-Id: I81e55c3c512f1dd6f49896f0c6b97a07d74fd8f5

Add Ogg/Opus playback support.

This uses libopusfile in order to be able to read .opus files and play
them back.

Writing/recording support is not present at this time.

ASTERISK-26409

Change-Id: I8815d23345108d8ca7c0bd640f6a1ce6b4f56955
(cherry picked from commit daee8bbd5209b4158bc1785eede845a26e6cbeaa)

sd_notify() is used to notify systemd of changes to the status of the
process. This allows the systemd daemon to know when the process
finished loading (and thus only start another program after Asterisk has
finished loading).

To use this, use a systemd unit with 'Type=notify' for Asterisk.

This commit also adds the function ast_sd_notify(), a wrapper around
sd_notify that does nothing if not built with systemd support.

Also adds support for libsystemd detection in the configure script.

Change-Id: Ied6a59dafd5ef331c5c7ae8f3ccd2dfc94be7811

The DPMA and g729a, silk, siren7 and siren14 codecs hosted at
http://downloads.digium.com/pub/telephony/ are now listed in the
"External" sections of the "Resource Modules" and "Codec Translators"
pages in menuselect.  Any that are selected will automatically be
downloaded and installed when "make install" is run.  Their LICENSE and
README (if avaialble) files will be installed to
ASTVARLIBDIR/documentation/thirdparty/<product_name>.

Example use with codecs:

The codecs/codecs.xml file is a menuselect style xml file that lists
the codecs to be included.  Their support levels are 'external', which
triggers the download and install, and defaultenabled is no.  Also
because codec_g729a is actually in a directory named codec_g729 on the
download server, the newly added 'member_data' element is used to
override the default of the directory name being the package name.  You
can use the 'directory_name' attribute to keep default base URL
(http://downloads.digium.com/pub/telephony/) but use the new directory,
or you use the 'remote_url' attribute to specify a full URL to the
download directory.  In this case, you must still follow the same
subdirectory naming conventions as that used for the packages located
at 'http://downloads.digium.com/pub/telephony'.

A new configure option '--with-externals-cache' was added and like
'--with-sounds-cache' it allows the installer to cache tarballs so
they're not downloaded every time.

To assist with the download and install process, each external package
now has a manifest.xml file that, among other things, contains a package
version and checksums for each file in the tarball.  The manifest is
saved to both the cache directory and ASTMODDIR and together with the
manifest.xml on the downloads site, tells the install scripts whether
a download and/or update is needed.

bash and xmlstarlet are required for downloader operation.  If they're
not installed, the external items in menuselect will be unavailable.

Change-Id: Id3dcf1289ffd3cb0bbd7dfab3cafbb87be60323a

ASTERISK-26217 #close

Change-Id: I1e45d8084683fab5f2b272bf35f4a149cea8b8d6

The pjsua and pjsystest apps are now built only if TEST_FRAMEWORK is set.
The python bindings are now built only if TEST_FRAMEWORK is set and a
python development package is installed.

libresample was also disabled.

ASTERISK-25993 #close
Reported-by: Joshua Colp

Change-Id: If4e91c503a02f113d5b71bc8b972081fa3ff6f03

Background here:
http://lists.digium.com/pipermail/asterisk-dev/2016-January/075266.html

From CHANGES:
 * To help insure that Asterisk is compiled and run with the same known
   version of pjproject, a new option (--with-pjproject-bundled) has been
   added to ./configure.  When specified, the version of pjproject specified
   in third-party/versions.mak will be downloaded and configured.  When you
   make Asterisk, the build process will also automatically build pjproject
   and Asterisk will be statically linked to it.  Once a particular version
   of pjproject is configured and built, it won't be configured or built
   again unless you run a 'make distclean'.

   To facilitate testing, when 'make install' is run, the pjsua and pjsystest
   utilities and the pjproject python bindings will be installed in
   ASTDATADIR/third-party/pjproject.

   The default behavior remains building with the shared pjproject
   installation, if any.

Building:

   All you have to do is include the --with-pjproject-bundled option on
   the ./configure command line (and remove any existing --with-pjproject
   option if specified).  Everything else is automatic.

Behind the scenes:

   The top-level Makefile was modified to include 'third-party' in the
   list of MOD_SUBDIRS.

   The third-party directory was created to contain any third party
   packages that may be needed in the future.  Its Makefile automatically
   iterates over any subdirectories passing on targets.

   The third-party/pjproject directory was created to house the pjproject
   source distribution.  Its Makefile contains targets to download, patch
   configure, generate dependencies, compile libs, apps and python bindings,
   sanitized build.mak and generate a symbols list.

   When bootstrap.sh is run, it automatically includes the configure.m4
   file in third-party/pjproject.  This file has a macro to download and
   conifgure pjproject and get and set PJPROJECT_INCLUDE, PJPROJECT_DIR
   and PJPROJECT_BUNDLED.  It also tests for the capabilities like
   PJ_TRANSACTION_GRP_LOCK by parsing preprocessor output as opposed to
   trying to compile.  Of course, bootstrap.sh is only run once and the
   configure file is incldued in the patch.

   When configure is run with the new options, the macro in configure.m4
   triggers the download, patch, conifgure and tests.  No compilation is
   performed at this time.  The downloaded tarball is cached in /tmp so
   it doesn't get downloaded again on a distclean.

   When make is run in the top-level Asterisk source directory, it will
   automatically descend all the subdirectories in third_party just as it
   does for addons, apps, etc.  The top-level Makefile makes sure that
   the 'third-party' is built before 'main' so that dependencies from the
   other directories are built first.

   When main does build, a new shared library (libasteriskpj) is created that
   links statically to the pjproject .a files and exports all their symbols.
   The asterisk binary links to that, just as it does with libasteriskssl.

   When Asterisk is installed, the pjsua and pjsystest apps, and the pjproject
   python bindings are installed in ASTDATADIR/third-party/pjproject.  This
   will facilitate testing, including running the testsuite which will be
   updated to check that directory for the pjsua module ahead of the system
   python library.

Modules should continue to depend on pjproject if they use pjproject APIs
directly.  They should not care about the implementation.  No changes to any
res_pjsip modules were made.

Change-Id: Ia7a60c28c2e9ba9537c5570f933c1ebcb20a3103

* Add 'check-alembic' target to root Makefile.
* Create build_tools/make_check_alembic to do the actual checks.

ASTERISK-25685

Change-Id: Ibb3cae7d1202ac23dc70b0f3b5801571ad46b004

clock_gettime() is, unfortunately, not portable. But I did like that
over our usual `ts.tv_nsec = tv.tv_usec * 1000` copy/paste code we
usually do when we want a timespec and all we have is ast_tvnow().

This patch adds ast_tsnow(), which mimics ast_tvnow(), but returns a
timespec. If clock_gettime() is available, it will use that. Otherwise
ast_tsnow() falls back to using ast_tvnow().

Change-Id: Ibb1ee67ccf4826b9b76d5a5eb62e90b29b6c456e

Created autoconf/ast_check_raii.m4: contains AST_CHECK_RAII which
checks compiler requirements for RAII:
gcc: -fnested-functions support
clang: -fblocks (and if required -lBlocksRuntime)
The original check was implemented in configure.ac and now has it's
own file. This function also sets C_COMPILER_FAMILY to either gcc or
clang for use by makefile

Created autoconf/ast_check_strsep_array_bounds.m4 (contains
AST_CHECK_STRSEP_ARRAY_BOUNDS):
which checks if clang is able to handle the optimized strsep & strcmp
functions (linux). If not, the standard libc implementation should be
used instead. Clang + the optimized macro's work with:
strsep(char *, char []), but not with strsepo(char *, char *).
Instead of replacing all the occurences throughout the source code,
not using the optimized macro version seemed easier

See 'define __strcmp_gc(s1, s2, l2) in bits/string2.h':
llvm-comment: Normally, this array-bounds warning are suppressed for
macros, so that unused paths like the one that accesses __s1[3] are
not warned about.  But if you preprocess manually, and feed the
result to another instance of clang, it will warn about all the
possible forks of this particular if statement. Instead of switching
of this optimization, another solution would be to run the preproces-
sing step with -frewrite-includes, which should preserve enough
information so that clang should still be able to suppress the diag-
nostic at the compile step later on.

See also "https://llvm.org/bugs/show_bug.cgi?id=20144"
See also "https://llvm.org/bugs/show_bug.cgi?id=11536"

Makefile.rules: If C_COMPILER_FAMILY=clang then add two warning
suppressions:
-Wno-unused-value
-Wno-parentheses-equality
In an earlier review (reviewboard: 4550 and 4554), they were deemed a
nuisace and less than benefitial.

configure.ac:
Added AST_CHECK_RAII() see earlier
Added AST_CHECK_STRSEP_ARRAY_BOUNDS() see earlier
Removed moved content

ASTERISK-24917
Change-Id: I12ea29d3bda2254ad3908e279b7effbbac6a97cb

This change adds an abstracted core DNS API which resembles the API described
here[1]. The API provides a pluggable mechanism for resolvers and also a
consistent view for records. Both synchronous and asynchronous queries are
supported.

This change also adds a res_resolver_unbound module which uses the libunbound
library to provide resolution.

Unit tests have also been written for all of the above to confirm the API and
functionality.

ASTERISK-24834 #close
Reported by: Matt Jordan

ASTERISK-24836 #close
Reported by: Matt Jordan

Review: https://reviewboard.asterisk.org/r/4474/
Review: https://reviewboard.asterisk.org/r/4512/

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+DNS+API


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433370 65c4cc65-6c06-0410-ace0-fbb531ad65f3

RAII_VAR, which is used extensively in Asterisk to manage reference counted
resources, uses a GCC extension to automatically invoke a cleanup function
when a variable loses scope. While this functionality is incredibly useful
and has prevented a large number of memory leaks, it also prevents Asterisk
from being compiled with clang.

This patch updates the RAII_VAR macro such that it can be compiled with clang.
It makes use of the BlocksRuntime, which allows for a closure to be created
that performs the actual cleanup.

Note that this does not attempt to address the numerous warnings that the clang
compiler catches in Asterisk.

Much thanks for this patch goes to:
* The folks on StackOverflow who asked this question and Leushenko for
  providing the answer that formed the basis of this code:
  http://stackoverflow.com/questions/24959440/rewrite-gcc-cleanup-macro-with-nested-function-for-clang
* Diederik de Groot, who has been extremely patient in working on getting this
  patch into Asterisk.

Review: https://reviewboard.asterisk.org/r/4370/

ASTERISK-24133
ASTERISK-23666
ASTERISK-20399
ASTERISK-20850 #close
Reported by: Diederik de Groot
patches:
  RAII_CLANG.patch uploaded by Diederik de Groot (License 6600)
........

Merged revisions 432807 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 432808 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@432809 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Because opus transcoding support cannot be included in the standard Asterisk
distribution, a few codec_opus implementations have popped up.  To make it
easier for people to drop in opus support in their own installations, this
patch adds configure checks for libopus.

Review: https://reviewboard.asterisk.org/r/4106/
........

Merged revisions 426234 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@426235 65c4cc65-6c06-0410-ace0-fbb531ad65f3

gcc on the ARM platform defaults 'char' to 'unsigned char' whereas Intel and
SPARC default to 'signed char'.  This is only an issue in the rare cases where
negative values are assigned to a 'char' but this this patch insures
compatibility by detecting platforms that default to 'unsigned' and adding an
'-fsigned-char' flag to _ASTCFLAGS.

If compiling for ARM (native or cross-compile) be sure to run ./bootstrap.sh
and ./configure to regenerate the build files.  You shouldn't have to do this
for Intel or SPARC.

Tested-by: George Joseph

Review: https://reviewboard.asterisk.org/r/4091/
........

Merged revisions 425964 from http://svn.asterisk.org/svn/asterisk/branches/12
........

Merged revisions 425965 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@425966 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch changes the install to only install the hook script if
DAHDI is enabled. It also adds the script to the uninstall task, and
moves the DAHDI_UDEV_HOOK_DIR variable so that it's not between the
_MAKEOPTS variables and their comment.

This allows installs which specify a --prefix to work normally, as
long as they don't enable DAHDI.

Review: https://reviewboard.asterisk.org/r/3972/
........

Merged revisions 423281 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@423283 65c4cc65-6c06-0410-ace0-fbb531ad65f3