As described in the issue, /tmp is not a suitable location for a
large amount of cached media files, since most distributions make
/tmp a RAM-based tmpfs mount with limited capacity.

I opted for a location that can be configured separately, as opposed
to using a subdirectory of spooldir, given the different storage
profile (transient files vs files that might stay there indefinitely).

This commit just makes the cache directory configurable, and changes
the default location from /tmp to /var/cache/asterisk.

ASTERISK-29143

Change-Id: Ic54e95199405abacd9e509cef5f08fa14c510b5d

Change-Id: I7d951e75ad2d472fa096647dfb55670b11105e23

This is a generic jenkinsfile to build Asterisk and optionally
perform one or more of the following:
 * Publish the API docs to the wiki
 * Run the Unit tests
 * Run Testsuite Tests

This job can be triggered manually from Jenkins or be triggered
automatically on a schedule based on a cron string.

Change-Id: Id9d22a778a1916b666e0e700af2b9f1bacda0852

If you do a "make all" when building Asterisk the xml documentation
produced will be missing certain AMI events where their
documentation is located not at the top of the c source file but
embedded further down next to the event's manager_event()
registration call.  See main/manager_mwi.c for an example.

"make full" does produce the correct documentation so we're changing
it in the build script.  A separate commit/issue will address the
problem with "make all".

ASTERISK-28507
Reported by: David Lee

Change-Id: I4a22635d6eef99eacecc0efb69e28360eebdb86c

Change-Id: I73ed4aef33a92f20080128aafc34e19fd4457196

Change-Id: I9f44a3d5085ea7880fad1a3883a4820907e29ea3
(cherry picked from commit 95213b01)

Change-Id: Ib4b6e4887695f230ea7a5b0c879b29fc5a13be4f
(cherry picked from commit d60f23ec)
(cherry picked from commit ce8a23fd)
(cherry picked from commit f0d1ce50)

The original clone depth of 10 was causing the need to rebase
changes whose parent was older than the 10 commits.  The clone
depth has been increased to 100.

Workspace cleanup was only happening for successful builds which
wasn't enough to keep the 8G workspace in-memory drives on the
docker slaves from filling up.  Now the workspaces are cleaned up
after every build regardless of success/failure.  If you need to
preserve builds temporarily, you can log into Jenkins/Manage
Jenkins/Configure System and change the CLEANUP_WS_* environment
variable for the job type you're troubleshooting to "FALSE".

Change-Id: I0d7366e87cea714e5dbc9488caf718802fce75ca

Change-Id: I52be64c8f6af2bbe15148a856d1f10cb113e1e94
(cherry picked from commit c6558e09)

To make throttling by label fully active, the "throttle" option
has to be specified with a specific label.

You can now specify "skip_gate" in the Gerrit comments when you
do a +2 code review to tell Jenkins not to actually run the
gate.  You'd do this if you plan to manually merge the change.

Also updated the "printenv" debug output to better sort multi-line
comments.

Change-Id: I4c0b1085acec4805f2ca207eebac50aad81f27e2

Originally, the eligible nodes for a job were labelled only by
"swdev-docker".  So basically any node could run any job.  We had
found that allowing a node to run more than 1 gate at a time was
problematic so we limited the nodes to processing 1 job at a time.
With the creation of the Asterisk 17 branches however, we now have
so many active branches that getting checks and gates through in
a timely manner is problematic when a node can run only 1 job
at a time.

Now the nodes are also labelled by the job type they can run.
For instance: "asterisk-check", "asterisk-gate", etc.  With the
"Throttle Concurrent Builds" plugin, we can now allow a node to
run more than 1 job BUT throttle by job type.  For instance:
  Allow 2 jobs but only 1 asterisk-gate at a time.
Now a node can run 2 checks or 1 check and 1 gate or 1 gate but
not 2 gates at a time.

Change-Id: I2032bf6afbcec5c341d9b852214c0c812d3d6db5

We don't support non-core modules for Certified releases but we
were enabling them for CI builds which was causing lots of test
failures.  Now we don't.

Change-Id: I0b3254c08a2479f3d39151690350cce5ce5ad766

We're at the point where there are enough Jenkins jobs for
Asterisk branches than even cleaned checkouts of Asterisk
will add up to more disk space than is available on the
in-memory workspace mount.  Since we archive all relevent
artifacts anyway, there's no need to keep the workspace
around after the job finishes, whether it succeeds or fails.

Change-Id: I1cd3b73ebb045a987df0f62526d152a510210c39

The testsuite actually needs the headers installed to run
it's self_test.

Change-Id: Ice41d331131b876ad4a9c056085fe6aac34b32b2

We were using the presence of /usr/lib64 to determine where
shared libraries should be installed.  This only existed on
Redhat based systems and was safe.  If it existed, use it,
otherwise use /usr/lib.

Unfortunately, Ubuntu 19 decided to create a /usr/lib64 BUT
NOT INCLUDE IT IN THE DEFAULT ld.so.conf.  So if anything is
installed there, it won't work.

The new method, just looks for $ID in /etc/os-release and if it's
centos or fedora, uses /usr/lib64 and if ubuntu, uses /usr/lib.

NOTE:  This applies only to the CI scripts.  Normal asterisk
build and install is not affected.

Change-Id: Iad66374b550fd89349bedbbf2b93f8edd195a7c3

One of the downaides of having things like test configuration
in the git repo is that it can't be changed at runtime.  You have
to create a review for the changes and merge it mefore it will
take effect.

This review moves the data currently held in
tests/CI/periodic-dailyTestGroups.json and
tests/CI/gateTestGroups.json into a Jenkins Config File attached
to the job definitions.  This allows us to alter it from the
Jenkins UI at runtime.  The original files stay in the repo
as documentation.

Change-Id: I14b9702f6285ce1fb2420287ba0e7d3b59109763

The new option disables dev mode, TEST_FRAMEWORK and
MALLOC_DEBUG making the build more production-like.

Change-Id: Ieb72497d4d91d5416684aaed702cc3f532099738

The recent upgrade of Gerrit to 2.16 elimiated referencing a
repository in a way the jenkinsfiles were relying on so
the URL references were changed to a more consistent and supported
format.

Change-Id: I2e8e3f213b9a96bb1b27665eca4a9a24bc49820e
(cherry picked from commit 5ce08457)

Some tests require Asterisk to execute scripts which
are stored in /tmp. When mount is used for tmpfs there
is no ability to allow scripts to be executed from
that location.

This change switches to using tmpfs which can be told
to allow executables to be run from /tmp.

Change-Id: I0e598ca2b76af1f7f2d29f0da7b1731a214a291a

This change makes it so that even if non-code changes
occur (such as commit message changing) unit tests
will still be run and result in a verification.

ASTERISK-28251

Change-Id: I6491fff7c93e5d5cd8e41054486968bf66c4f608

* Added ---no-configure, --no-menuselect, --no-make and --no-alembic
  options that prevent those actions from being performed.  Useful
  for testing and re-running portions of the build after fixing
  earlier failures.

* Added "set -e" to abort the script on command failure.
  Not sure why this wasn't there in the first place.

* Fixed a few echos that were redirecting to stderr when they shouldn't
  have been.

* Catch more alembic failures by actually trying to generate the SQL.

Change-Id: I9f395fa4e9254be7299e7c1014f1a13db78faffb

The job timeouts were hard coded in the jenkinsfiles which
means changes had to go through gerrit.  Now they are taken
from the following environment variables (and their defaults) that
can be set in Jenkins configuration...

TIMEOUT_GATES =      "60 MINUTES"
TIMEOUT_DAILIES =    "3 HOURS"
TIMEOUT_REF_DEBUG =  "24 HOURS"
TIMEOUT_UNITTESTS =  "30 MINUTES"

Change-Id: I673a551c1780bf665a3bc160b245da574aa4bbab

Change-Id: Ida29d70d48d5f39aabf0b25c66b51f79324a8cba

Change-Id: I0566d81b0852f22066cd76d58eae5f1fda5602aa
(cherry picked from commit 73efe864)

The testsuite can now use a user-specified work directory for
all it's temp files.  This allows the docker containers to use
a tmpfs backed directory for the temp files instead of it's
own write-layer image.

* runTestsuite.sh now accepts a --work-dir command line argument
  that gets exported as AST_WORK_DIR before running the testsuite.

* gates.jenkinsfile now specifies --work-dir to be
  <testsuite_dir>/astroot.

Since the Asterisk CI docker hosts now mount /srv/jenkins/workspace
on a tmpfs, asterisk should be compiled and the testsuite run all in
memory.

Change-Id: If5ee905a15821296c355bb84cda38950ad8edc45
(cherry picked from commit a335f4c9)

There seems to be a race condition between starting the asterisk
daemon and attempting to use 'asterisk -r' that can cause the
control socket file to not be created.  Since all of the Jenkins
slaves have 'expect' installed, the runUnittests script can use
it to start asterisk in the forground and issue the commands
interactively.  This is much more reliable and it can also make
startup errors more visible since they'll be in the Jenkins console
output.

If 'expect' isn't installed, the original daemon/asterisk -r
process is used.

Also added a "core show settings" before running the tests
and added "notice,warning,error" to the console log.

Change-Id: Idd656085f854afede813ac241b9e312b31358160

Change-Id: I2f252e0f8c7f1a6328438fbd2be5d6574b7dfa5b

Change-Id: Ide83574dc957bc1df28e30a69079140050dfc35f

Change-Id: I8246a0147df8d821fbbcabc1db1887104b8bedc4

Fix redirection to /dev/null of cleanup commands.  The '2' was being
interpreted as part of the command instead of part of the redirect.

Change-Id: I2e3a591b165e0288c4b82b9ef475fdfd5392a90a

Also make the shebang in publishAsteriskDocs.sh the first line.

Change-Id: I3fdd6f22e652e4fb5b5fe85df46fa34eb6d0cf08

Change-Id: Ife9a6879da63a56e5b8348a2024eeed4e7b1615b

The default is 600 seconds.
Also added timeouts to the *TestGroups.json files.

Change-Id: I8ab6a69e704b6a10f06a0e52ede02312a2b72fe0

Change-Id: I30024515e5b00a5044fd39fbff27d818f016b719

Can't do anonymous http checkout from Security-testsuite.
Need to use same credentials as the gerrit review checkout.

Change-Id: I87af68c995cb8926f5e87f9af245600d76984f05

This ensures that binary modules are avoided in the master branch even
if BRANCH_NAME is not set.

Change-Id: I79162d2063f22fa9d6b31fde4827ace2dd5bf0da

This new option can be passed for ./configure or
./tests/CI/buildAsterisk.sh to prevent download/install of binary
modules.

Normally enabling the categories MENUSELECT_CODECS or MENUSELECT_RES
will result in binary modules being enabled even if the build target is
incompatible with those modules.  This includes CI scripts which enable
categories before disabling specific modules.

If more binary modules are offered in the future this will help avoid
accidentally downloading them if unwanted or incompatible.  Adding a
binary module will only require creating a new menuselect entry similar
to the existing ones, it will not be necessary to modify the CI scripts.

Change-Id: I6b1bd1c75a2e48f05b8b8a45b7a7a2d00a079166

Change-Id: I5e1d4a09e58b92b541bc8ed6f9e10e54c4e5101f

Change-Id: I7729ecceedceb12f52bf18dae259846aa1d993b3

If the review to be tested is in a project with restricted access,
we need to use the jenkins user's gerrit https credentials when we
do the checkout or the checkout will fail.

Change-Id: I9dc9994763c5ebfeb9f1cff60fb53f6902b7fd5f