We use a lot res_calendar, we are very happy with that, especially
because you use libical, the almost alone opensource library that
supports really ical format with all types of recurrency.

Nevertheless, some features are missed for our business use cases.

This first patch adds a new option in calendar.conf:
fetch_again_at_reload. Be my guest for a better name.

If it's true, when you'll launch "module reload res_calendar.so",
Asterisk will download again the calendar.

The business use case is that we have a WebUI with a scheduler planner,
we know when the calendars are modified.

For now, we need to define 1 minute of timeout to have a chance that
our user doesn't wait too long between the modification and the real
test.  But it generates a lot of useless HTTP traffic.


ASTERISK-26422 #close

Change-Id: I384b02ebfa42b142bbbd5b7221458c7f4dee7077

The pooling, shared_connection, limit, and idlecheck options
are no longer used in res_odbc.

ASTERISK-26389

Change-Id: I2fde7b467d01f9d1c82cc0a339bb4f7e1dd6bbe6

* Make 'cdrzone=UTC' work properly.
* Fix the documentation of cdr_mysql.conf: it's cdrzone and not timezone

ASTERISK-26359 #close

Change-Id: I2a6f67b71bbbe77cac31a34d0bbfb1d67c933778

This implements the chan_sip legacy_useroption_parsing option but with a
better name.

* Made the caller-id number and redirecting number strings obtained from
incoming SIP URI user fields always truncated at the first semicolon.
People don't care about anything after the semicolon showing up on their
displays even though the RFC allows the semicolon.

ASTERISK-26316 #close
Reported by: Kevin Harwell

Change-Id: Ib42b0e940dd34d84c7b14bc2e90d1ba392624f62

This patch add config to pjsip by endpoint.
;preferred_codec_only=yes
; Respond to a SIP invite with the single most preferred codec
; rather than advertising all joint codec capabilities. This
; limits the other side's codec choice to exactly what we prefer.

ASTERISK-26317 #close
Reported by: AaronAn
Tested by: AaronAn

Change-Id: Iad04dc55055403bbf5ec050997aee2dadc4f0762

Change-Id: Ie627def9604ae30abd80754f9e6f09874825aec6

When 'tlsclientmethod' is not specified in sip.conf, chan_sip uses the OpenSSL
SSLv23_method. This was documented incorrectly in the file sip.conf.sample.

SSLv23_method got its name in the 90s. Today, with OpenSSL 1.0.2, this method
enables (just) the secure TLSv1.0 and TLSv1.2. Or stated differently, that
function should have been called 'secure_method' or 'automatic_method' back in
the 90s.

Consequently please, specify 'tlsclientmethod=tlsv1' in your sip.conf only if
you face a server which has problems like not falling back to TLSv1.0
automatically.

ASTERISK-24425

Change-Id: I502ce6146b4504cadfd3973af8d6ec3994f54fa3

contact_user, when specified on an endpoint, will override the user
portion of the Contact header on outgoing requests.

Change-Id: Icd4ebfda2f2e44d3ac749d0b4066630e988407d4

The Exchanging Device and Mailbox States could not working
if the Entity ID (EID) is not set manually and can't be obtained
from ethernet interface.

This patch replaces debug message to warning
and addes missing description about option 'entityid' to
asterisk.conf.sample.

With this patch the asterisk also:
(1) decline loading the modules which won't work without EID:
    res_corosync and res_pjsip_publish_asterisk.
(2) warn if EID is empty on loading next modules:
    pbx_dundi, res_xmpp

Starting with v197 systemd/udev will automatically assign "predictable"
names for all local Ethernet interfaces.
This patch also addes some new ethernet prefixes "eno" and "ens".

ASTERISK-26164 #close

Change-Id: I72d712f1ad5b6f64571bb179c5cb12461e7c58c6

ASTERISK-26246 #close

Change-Id: Id673b9786389f9d2a87f638ce1a25161f5f31657

Make it clear that we're talking about device state hints and add
an entry to the sample config.

Change-Id: Iaef58ffb960191a21b713e8e0b51ce1fcd47e433

The PJSIP taskprocessors could be overflowed on startup
if there are many (thousands) realtime endpoints
configured with unsolicited mwi.
The PJSIP stack could be totally unresponsive for a few minutes
after boot completed.

This patch creates a separate PJSIP serializers pool for mwi
and makes unsolicited mwi use serializers from this pool.
This patch also adds 2 new global options to tune taskprocessor
alert levels: 'mwi_tps_queue_high' and 'mwi_tps_queue_low'.

This patch also adds new global option 'mwi_disable_initial_unsolicited'
to disable sending unsolicited mwi to all endpoints on startup.
If disabled then unsolicited mwi will start processing
on next endpoint's contact update.

ASTERISK-26230 #close

Change-Id: I4c8ecb82c249eb887930980a800c9f87f28f861a

On heavy loaded system with IMAP or DB storage,
'app_voicemail' taskprocessor queue could reach 500 scheduled tasks.
It could happen when the IMAP or DB server dies or is unreachable.
It could happen on startup when there are many (thousands)
realtime endpoints configured with unsolicited mwi.
If the taskprocessor queue reaches the high water level
then the alert is triggered and pjsip stops processing new requests
until the queue reaches the low water level to clear the alert.

This patch adds 2 new 'general' configuration options
to tune taskprocessor alert levels:
'tps_queue_high' - Taskprocessor high water alert trigger level.
'tps_queue_low' - Taskprocessor low water clear alert level

ASTERISK-26229 #close

Change-Id: I766294fbffedf64053c0d9ac0bedd3109f043ee8

Change-Id: Idf97e3a72f1edc5fca58f2fa7b20785922be0cae

Change-Id: I11139a4a95df34e223ba622aa6227e33ab8f6c38

The new option allows the channel driver's faxdetect option to timeout on
a call after the specified number of seconds into a call.  The new feature
is disabled if the timeout is set to zero.  The option is disabled by
default.

* Don't clear dsp_features after passing them to the dsp code in
my_pri_ss7_open_media().  We should still remember them especially for the
new faxdetect_timeout option.

ASTERISK-26214
Reported by: Richard Mudgett

Change-Id: Ieffd3fe788788d56282844774365546dce8ac810

The new endpoint option allows the PJSIP channel driver's fax_detect
endpoint option to timeout on a call after the specified number of
seconds into a call.  The new feature is disabled if the timeout is set
to zero.  The option is disabled by default.

ASTERISK-26214
Reported by: Richard Mudgett

Change-Id: Id5a87375fb2c4f9dc1d4b44c78ec8735ba65453d

Following the principle of least surprise, we should not be sending
massive numbers of PJSIP and RTCP HEP packets out into the ether to some
only-slightly-random IP address. Having 'enabled' set to 'no' in the
sample configuration file should prevent this from happening for those
who run 'make samples'.

ASTERISK-26159 #close

Change-Id: I1753a64ca83a3442a6ebdc31061f8185c062d9b1

This patch removes the following modules:
 - pbx_functions: It never existed.
 - res_pjsip_log_forwarder: It no longer exists.
 - res_hep_pjsip: The base HEP module wasn't loaded, and most basic PBXs
                  aren't going to be installing HOMER
 - res_pjsip_phoneprov_provider: The basic res_phoneprov module isn't
                  loaded, and we aren't configured to make use of the
                  module

Change-Id: Id91f68cae7c9c8c3d370029fe1268cb51e4ff5a5

Testing has shown that our usage of UnixODBC is problematic
due to bugs within UnixODBC itself as well as the heavy weight
cost of connecting and disconnecting database connections, even
when pooling is enabled.

For users of UnixODBC 2.3.1 and earlier crashes would occur due
to insufficient protection of the disconnect operation. This was
fixed in UnixODBC 2.3.2 and above.

For users of UnixODBC 2.3.3 and higher a slow-down would occur
under heavy database use due to repeated connection establishment.
A regression is present where on each connection the database
configuration is cached again, with the cache growing out of
control.

The connection pool implementation present in this change helps
to mitigate these issues by reducing how much we connect and
disconnect database connections. We also solve the issue of
crashes under UnixODBC 2.3.1 by defaulting the maximum number of
connections to 1, returning us to the previous working behavior.
For users who may have a fixed version the maximum concurrent
connection limit can be increased helping with performance.

The connection pool works by keeping a list of active connections.
If the connection limit has not been reached a new connection is
established. If the connection limit has been reached then the
request waits until a connection becomes available before
continuing.

ASTERISK-26074 #close
ASTERISK-26054 #close

Change-Id: I6774bf4bac49a0b30242c76a09c403d2e856ecff

ASTERISK-20527 #close

Change-Id: I659cf7f00836a09d09d146ad226a40477d731239

Add the option 'enable_callee_prompt' to followme.conf. Enabled by
default. If disabled, a callee is not prompted to accept or reject
the forwarded call.

ASTERISK-26064 #close

Change-Id: I0a8b19d4cf95c86a07c992813babb9e4a4acfff5
Signed-off-by: Tzafrir Cohen <tzafrir.cohen@xorcom.com>

func_odbc was changed in Asterisk 13.9.0
to make func_odbc use a single database connection per DSN
because of reported bug ASTERISK-25938
with MySQL/MariaDB LAST_INSERT_ID().

This is drawback in performance when func_odbc is used
very often in dialplan.

Single database connection should be optional.

ASTERISK-26010

Change-Id: I7091783a7150252de8eeb455115bd00514dfe843

A ':' is not a valid token for starting a comment.

Change-Id: I123592d93a83d1bdde3e352822881eb9da85e5ad

At one point in time, it seemed like a good idea to use the Asterisk
channel name as the HEP correlation UUID. In particular, it felt like
this would be a useful identifier to tie PJSIP messages and RTCP
messages together, along with whatever other data we may eventually send
to Homer. This also had the benefit of keeping the correlation UUID
channel technology agnostic.

In practice, it isn't as useful as hoped, for two reasons:
1) The first INVITE request received doesn't have a channel. As a
   result, there is always an 'odd message out', leading it to be
   potentially uncorrelated in Homer.
2) Other systems sending capture packets (Kamailio) use the SIP Call-ID.
   This causes RTCP information to be uncorrelated to the SIP message
   traffic seen by those capture nodes.

In order to support both (in case someone is trying to use res_hep_rtcp
with a non-PJSIP channel), this patch adds a new option, uuid_type, with
two valid values - 'call-id' and 'channel'. The uuid_type option is used
by a module to determine the preferred UUID type. When available, that
source of a correlation UUID is used; when not, the more readily available
source is used.

For res_hep_pjsip:
 - uuid_type = call-id: the module uses the SIP Call-ID header value
 - uuid_type = channel: the module uses the channel name if available,
                        falling back to SIP Call-ID if not
For res_hep_rtcp:
 - uuid_type = call-id: the module uses the SIP Call-ID header if the
                        channel type is PJSIP and we have a channel,
                        falling back to the Stasis event provided
                        channel name if not
 - uuid_type = channel: the module uses the channel name

ASTERISK-25352 #close

Change-Id: Ide67e59a52d9c806e3cc0a797ea1a4b88a00122c

* No need to set language in a miniml configuration. 'en' will do just
  fine.
* It would be useful to have an example of setting it to a different
  language.
* Setting the documentation language explicitly is likewise not
  required. Setting it to a different value is not common. At least
  until there is a set of translated documentation.

Change-Id: I94d91ea34e129925f25af81ef8dc0906fb568cb7
Signed-off-by: Tzafrir Cohen <tzafrir.cohen@xorcom.com>

Don't suggest users to use debug level 5, which spews (usually
non-useful) debug information. Reduce the suggestion to (an
arbitrarily-selected) level 2.

Change-Id: Ib53195f78945970956ff59ef13fa89b90e0fcd60
Signed-off-by: Tzafrir Cohen <tzafrir.cohen@xorcom.com>

Note the default of remmed-out options. To clarify that those values are
not the defaults.

Change-Id: I849c29b7a710f0abc37355fcb5bfee335ae30738
Signed-off-by: Tzafrir Cohen <tzafrir.cohen@xorcom.com>

A minimal configuration does not need to explicitly spell out the
directories. The built-in defaults will do just fine. In many cases
they are wrong.

Change-Id: Id1a671e5c5e9923765a4156b57f9f7e263fdd26c
Signed-off-by: Tzafrir Cohen <tzafrir.cohen@xorcom.com>

This patch allows for having app_confbridge register the name of the
conference as an extension into a specific context, similar to
regcontext for chan_sip.  This variant is not quite as involved as the
one in chan_sip and doesn't allow for multiple contexts or custom
extensions, you can only specify the context and the conference name
will always be used as the extension to register.

ASTERISK-25989 #close

Change-Id: Icacf94d9f2b5dfd31ef36f6cb702392619a7902f

Note: When packagers use these files (as an example) the paths are never
really used when they are split using '='.

Note: Thirdparty applications will also have trouble parsing the file when
expecting '=>'.

Change-Id: I0ada647f588e81f023fb1333ca15a1a333fd6004

A feature of chan_sip that service providers relied upon was the ability to
identify by the Authorization username.  This is most often used when customers
have a PBX that needs to register rather than identify by IP address.  From my
own experiance, this is pretty common with small businesses who otherwise
don't need a static IP.

In this scenario, a register from the customer's PBX may succeed because From
will usually contain the PBXs account id but an INVITE will contain the caller
id.  With nothing recognizable in From, the service provider's Asterisk can
never match to an endpoint and the INVITE just stays unauthorized.

The fixes:

A new value "auth_username" has been added to endpoint/identify_by that
will use the username and digest fields in the Authorization header
instead of username and domain in the the From header to match an endpoint,
or the To header to match an aor.  This code as added to
res_pjsip_endpoint_identifier_user rather than creating a new module.

Although identify_by was always a comma-separated list, there was only
1 choice so order wasn't preserved.  So to keep the order, a vector was added
to the end of ast_sip_endpoint.  This is only used by res_pjsip_registrar
to find the aor.  The res_pjsip_endpoint_identifier_* modules are called in
globals/endpoint_identifier_order.

Along the way, the logic in res_pjsip_registrar was corrected to match
most-specific to least-specific as res_pjsip_endpoint_identifier_user does.

The order is:

username@domain
username@domain_alias
username

Auth by username does present 1 problem however, the first INVITE won't have
an Authorization header so the distributor, not finding a match on anything,
sends a securty_alert.  It still sends a 401 with a challenge so the next
INVITE will have the Authorization header and presumably succeed.  As a result
though, that first security alert is actually a false alarm.

To address this, a new feature has been added to pjsip_distributor that keeps
track of unidentified requests and only sends the security alert if a
configurable number of unidentified requests come from the same IP in a
configurable amout of time.  Those configuration options have been added to
the global config object.  This feature is only used when auth_username
is enabled.

Finally, default_realm was added to the globals object to replace the hard
coded "asterisk" used when an endpoint is not yet identified.

The testsuite tests all pass but new tests are forthcoming for this new
feature.

ASTERISK-25835 #close
Reported-by: Ross Beer

Change-Id: I30ba62d208e6f63439600916fcd1c08a365ed69d

This patch added new global pjsip option 'disable_multi_domain'.
Disabling Multi Domain can improve Realtime performance by reducing
number of database requests.

ASTERISK-25930 #close

Change-Id: I2e7160f3aae68475d52742107949a799aa2c7dc7

Option was removed in commit 7f883ef4

ASTERISK-25927 #close

Change-Id: I92f9b0196d9fc41d1d58354c07340c465ef1fcf8

res_pjsip_mwi was missing the chan_sip "vmexten" functionality which adds
the Message-Account header to the MWI NOTIFY.  Also, specifying mailboxes
on endpoints for unsolicited mwi and on aors for subscriptions required
that the admin know in advance which the client wanted.  If you specified
mailboxes on the endpoint, subscriptions were rejected even if you also
specified mailboxes on the aor.

Voicemail extension:
* Added a global default_voicemail_extension which defaults to "".
* Added voicemail_extension to both endpoint and aor.
* Added ast_sip_subscription_get_dialog for support.
* Added ast_sip_subscription_get_sip_uri for support.

When an unsolicited NOTIFY is constructed, the From header is parsed, the
voicemail extension from the endpoint is substituted for the user, and the
result placed in the Message-Account field in the body.

When a subscribed NOTIFY is constructed, the subscription dialog local uri
is parsed, the voicemail_extension from the aor (looked up from the
subscription resource name) is substituted for the user, and the result
placed in the Message-Account field in the body.

If no voicemail extension was defined, the Message-Account field is not added
to the NOTIFY body.

mwi_subscribe_replaces_unsolicited:
* Added mwi_subscribe_replaces_unsolicited to endpoint.

The previous behavior was to reject a subscribe if a previous internal
subscription for unsolicited MWI was found for the mailbox.  That remains the
default.  However, if there are mailboxes also set on the aor and the client
subscribes and mwi_subscribe_replaces_unsolicited is set, the existing internal
subscription is removed and replaced with the external subscription.  This
allows an admin to configure mailboxes on both the endpoint and aor and allows
the client to select which to use.

ASTERISK-25865 #close
Reported-by: Ross Beer

Change-Id: Ic15a9415091760539c7134a5ba3dc4a6a1217cea

There were a number of places in the res_pjsip stack that were getting
all endpoints or all aors, and then filtering them locally.

A good example is pjsip_options which, on startup, retrieves all
endpoints, then the aors for those endpoints, then tests the aors to see
if the qualify_frequency is > 0.  One issue was that it never did
anything with the endpoints other than retrieve the aors so we probably
could have skipped a step and just retrieved all aors. But nevermind.

This worked reasonably well with local config files but with a realtime
backend and thousands of objects, this was a nightmare.  The issue
really boiled down to the fact that while realtime supports predicates
that are passed to the database engine, the non-realtime sorcery
backends didn't.

They do now.

The realtime engines have a scheme for doing simple comparisons. They
take in an ast_variable (or list) for matching, and the name of each
variable can contain an operator.  For instance, a name of
"qualify_frequency >" and a value of "0" would create a SQL predicate
that looks like "where qualify_frequency > '0'".  If there's no operator
after the name, the engines add an '=' so a simple name of
"qualify_frequency" and a value of "10" would return exact matches.

The non-realtime backends decide whether to include an object in a
result set by calling ast_sorcery_changeset_create on every object in
the internal container.  However, ast_sorcery_changeset_create only does
exact string matches though so a name of "qualify_frequency >" and a
value of "0" returns nothing because the literal "qualify_frequency >"
doesn't match any name in the objset set.

So, the real task was to create a generic string matcher that can take a
left value, operator and a right value and perform the match. To that
end, strings.c has a new ast_strings_match(left, operator, right)
function.  Left and right are the strings to operate on and the operator
can be a string containing any of the following: = (or NULL or ""), !=,
>, >=, <, <=, like or regex.  If the operator is like or regex, the
right string should be a %-pattern or a regex expression.  If both left
and right can be converted to float, then a numeric comparison is
performed, otherwise a string comparison is performed.

To use this new function on ast_variables, 2 new functions were added to
config.c.  One that compares 2 ast_variables, and one that compares 2
ast_variable lists.  The former is useful when you want to compare 2
ast_variables that happen to be in a list but don't want to traverse the
list.  The latter will traverse the right list and return true if all
the variables in it match the left list.

Now, the backends' fields_cmp functions call ast_variable_lists_match
instead of ast_sorcery_changeset_create and they can now process the
same syntax as the realtime engines.  The realtime backend just passes
the variable list unaltered to the engine.  The only gotcha is that
there's no common realtime engine support for regex so that's been noted
in the api docs for ast_sorcery_retrieve_by_fields.

Only one more change to sorcery was done...  A new config flag
"allow_unqualified_fetch" was added to reg_sorcery_realtime.
"no": ignore fetches if no predicate fields were supplied.
"error": same as no but emit an error. (good for testing)
"yes": allow (the default);
"warn": allow but emit a warning. (good for testing)

Now on to res_pjsip...

pjsip_options was modified to retrieve aors with qualify_frequency > 0
rather than all endpoints then all aors.  Not only was this a big
improvement in realtime retrieval but even for config files there's an
improvement because we're not going through endpoints anymore.

res_pjsip_mwi was modified to retieve only endpoints with something in
the mailboxes field instead of all endpoints then testing mailboxes.

res_pjsip_registrar_expire was completely refactored.  It was retrieving
all contacts then setting up scheduler entries to check for expiration.
Now, it's a single thread (like keepalive) that periodically retrieves
only contacts whose expiration time is < now and deletes them.  A new
contact_expiration_check_interval was added to global with a default of
30 seconds.

Ross Beer reports that with this patch, his Asterisk startup time dropped
from around an hour to under 30 seconds.

There are still objects that can't be filtered at the database like
identifies, transports, and registrations.  These are not going to be
anywhere near as numerous as endpoints, aors, auths, contacts however.

Back to allow_unqualified_fetch.  If this is set to yes and you have a
very large number of objects in the database, the pjsip CLI commands
will attempt to retrive ALL of them if not qualified with a LIKE.
Worse, if you type "pjsip show endpoint <tab>" guess what's going to
happen? :)  Having a cache helps but all the objects will have to be
retrieved at least once to fill the cache.  Setting
allow_unqualified_fetch=no prevents the mass retrieve and should be used
on endpoints, auths, aors, and contacts.  It should NOT be used for
identifies, registrations and transports since these MUST be
retrieved in bulk.

Example sorcery.conf:

[res_pjsip]
endpoint=config,pjsip.conf,criteria=type=endpoint
endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error

ASTERISK-25826 #close
Reported-by: Ross Beer
Tested-by: Ross Beer

Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67

* Remove duplicate res_parking.conf courtesytone config option
documentation.

ASTERISK-24596 #close
Reported by:  Philip Correia

ASTERISK-24605
Reported by:  Philip Correia
Patches:
      call_park_app_doc.patch (license #6672) patch uploaded by Philip Correia

Change-Id: I90a92a891c6494dc08173e675856afcc4764c5b5

The 'reload' mechanism actually involves closing the underlying
socket and calling the appropriate udp, tcp or tls start functions
again.  Only outbound_registration, pubsub and session needed work
to reset the transport before sending requests to insure that the
pjsip transport didn't get pulled out from under them.

In my testing, no calls were dropped when a transport was changed
for any of the 3 transport types even if ip addresses or ports were
changed. To be on the safe side however, a new transport option was
added (allow_reload) which defaults to 'no'.  Unless it's explicitly
set to 'yes' for a transport, changes to that transport will be ignored
on a reload of res_pjsip.  This should preserve the current behavior.

Change-Id: I5e759850e25958117d4c02f62ceb7244d7ec9edf

Previously you could add [!dnid] to the SIP dial string to alter the To:
header. This change allows you to alter the From header as well.

SIP dial string extra options now look like this:

    [![touser[@todomain]][![fromuser][@fromdomain]]]

INCOMPATIBLE CHANGE: If you were using an exclamation mark in your To:
header, that is no longer possible.

ASTERISK-25803 #close

Change-Id: I2457e9ba7a89eb1da22084bab5a4d4328e189db7

Warnings and errors in the pjproject libraries are generally handled by
Asterisk.  In many cases, Asterisk wouldn't even consider them to be warnings
or errors so the messages emitted by pjproject directly are either superfluous
or misleading.  A good exampe of this are the level-0 errors pjproject emits
when it can't open a TCP/TLS socket to a client to send an OPTIONS.  We don't
consider a failure to qualify a UDP client an "ERROR", why should a TCP/TLS
client be treated any differently?

A config file for res_pjproject has bene added (pjproject.conf) and a new
log_mappings object allows mapping pjproject levels to Asterisk levels
(or nothing).  The defaults if no pjproject.conf file is found are the same
as those that were hard-coded into res_pjproject initially: 0,1 = LOG_ERROR,
2 = LOG_WARNING, 3,4,5 = LOG_DEBUG<level>

Change-Id: Iba7bb349c70397586889b8f45b8c3d6c6c8c3898