11.10.2022

TIPS for rpm, rpmbuild, yum

The main reason for me to use rpmbuild when i compile and install any software is that you can easily install and remove all files. In “make” case some time you can not do that by command “make remove”. Also when you are using “yum install” than installed libraries can be used by other software for solve dependencies.

Check installed files for certain package:

rpm -ql ffmpeg-libs

TIPs for create spec files:
(rus) https://blog.korphome.ru/2014/11/18/centos-собираем-пакеты-при-помощи-rpmbuild/
(eng) https://rpm-packaging-guide.github.io/#files

7.10.2022

Rtpengine. Opus. Ilbc.

I am using ilbc to make calls with mobile applications. As we know ilbc is old codec, all tests,table and pictures all over the net make us feel as ilbc most worse then opus. because opus is faster, more quality e.t.c.

Seems… my opinion is different, if you want good quality and minimum bandwidth out of box then use ilbc – it’s not problem, it will have 23kb bandwidth for one side. Audio bandwidth up to 4kHz so voice will be good enough for conversation. But there is no way to reduce bitrate and bandwidth with ilbc, so let’s try to implement opus.

also, converting 48kHz(sample rate) files with libopus is slowest then with ilbc in most of cases. Be noted that you can not use opus with 8kHz(sample rate) in default configuration by rtpengine only 48kHz is supported.

Components used for testing:

  • centos 7: iftop, tcpdump
  • custom ffmpeg 4.2.7,
  • opus 1.3.2,
  • opus-tools: opusenc, opusrtp
  • rtpengine 11.0.1.5,
  • microsip 3,
  • sipp 3.6.
  • clumsy ( simulate bad network on windows)

I will start from end, maybe it will be helpful for someone. What was my aim, i wanted to use packet loss, fec, speech mode, low bandwidth from opus codec.
Success:
* low bandwidth with 8kb\s bitrate (13kb\s actual) and 40ms frame duration.
Failed:
* packet_loss (no way to understand if it really works, real test does not show that it helps)
* fec, ( same as packet loss)
* speech mode, (take more cpu resources when encoding without real result)

How to add opus support into rtpengine.

For encoding\decoding opus rtpeginge using ffmpeg library. so you have to be sure that libopus is present with ffmpeg. you can do that with: “ffmpeg -h encoder=libopus” if you don’t see: “Codec ‘libopus’ is not recognized by FFmpeg.” then seems ffmpeg have opus with libopus encoder\decoder.

How to set parameters for opus codec:

when you do rtpengine_offer use this: as one of params:
codec-transcode-opus/48000/2/8000/40/maxaveragebitrate–8000;maxplaybackrate–12000;useinbandfec–1;ptime–40;maxptime–40/ar-48000,b–8000
where is :
48000 – sample rate in SDP
2 – channels (default for opus)
8000 (b\s) – bitrate for codec implement on encoder side
40 (ms) – frame duration (should affect on encoder side, but you have a=ptime 20 in SDP, codec will work on 20 ms)
“maxaveragebitrate–8000;maxplaybackrate–12000;useinbandfec–1;ptime–40;maxptime–40” there are a=fmtp parameters into SDP, you can check what it means in RFC.
“ar–48000,b–8000” – codec individual options you can take a look ffmpeg docs to check what you can use. For some reason individual options for opus codec like packet_loss can not be set by this logic, you have to set it inside codeclib.c in rtpengine source code . for example “

if (enc->ptime > 0 ) {
            codeclib_set_av_opt_int(enc, "frame_duration", enc->ptime);
            codeclib_set_av_opt_int(enc, "packet_loss", 5);
            codeclib_set_av_opt_int(enc, "fec", 1);
            codeclib_set_av_opt_int(enc, "application", 2048);
}


issues: when you set 40 ms frame_duration for opus and you have not any a=ptime 40 in SDP towards to destination peer, peer will not send stream with 40 ms frame_duration, maybe there is a bug into Microsip. Using ptime and maxptime into codec options – not helps.
so, to avoid this i did add ptime=40 as rtpengine_offer parameter and add little fix to codeclib.c to make 40ms default ptime for opus codec.

How to check speed of converting with libopus

you need any music input file, for example any.wav. then you may try to use
ffmpeg -i madonna-48k.wav -c libopus -ab 18000 madonna.opus
it will convert wav file to opus with bitrate 18k\s
as result you will see some data ended with
size= 82kB time=00:00:39.83 bitrate= 16.8kbits/s speed= 136x
also you can convert it with libilbc encoder:
ffmpeg -i madonna-48k.wav -c libilbc -ar 8000 -ab 18000 madonna.lbc
you will see:
size= 74kB time=00:00:39.84 bitrate= 15.2kbits/s speed= 170x

to be continued….

16.08.2022

Oracle Centos 8. Rtpengine with all codecs supported.

As result of this instruction you will have all this codecs supported in your centos 8 installations.

                PCMA: fully supported
                PCMU: fully supported
                G723: fully supported
                G722: fully supported
                G729: fully supported
               G729a: fully supported
               speex: fully supported
                 GSM: fully supported
                iLBC: fully supported
                opus: fully supported
                 AMR: fully supported
              AMR-WB: fully supported
     telephone-event: fully supported
                  CN: fully supported

Synopsis:

RPMS, build and install scripts: git clone https://bitbucket.org/yooxy/centos8-rtpengine10-all-codecs.git

This instruction will give you RTPENGINE for Centos 7 and Centos 8 withh all codecs. RPM packages in RPMS dir are ready for install. But also you have rpmbuild-rtpengine.el7 and rpmbuild-rtpengine.el8 to compile it on your system in automatically way.

if you start to compiling on new system, then everything should go fine after you type sh rpmbuild-rtpengine.el7.

IF you work on production system , then check files you are running before start due to you may to install unnecessary packets or kernels. 

To build rtpengine with all codecs (g729,AMR,opus,iLBC, GSM) on Centos 8:

cd ~
git clone https://bitbucket.org/yooxy/centos8-rtpengine10-all-codecs.git
sh rpmbuild-rtpengine.el8
cd ~/rpmbuild/RPMS/
dnf install noarch/ngcp-rtpengine-dkms-10.5.1.3+0~mr10.5.1.3-1.el8.noarch.rpm x86_64/ngcp-rtpengine-kernel-10.5.1.3+0~mr10.5.1.3-1.el8.x86_64.rpm x86_64/ngcp-rtpengine-10.5.1.3+0~mr10.5.1.3-1.el8.x86_64.rpm

Your RPMs ready for install in ~/root/rpmbuild/RPMS


To install rtpengine without build 10.5 run “sh install-rtpengine.el7”

18.06.2022

Auth SIP manual

How to md5 auth SIP client manually if you have access to DB with passwords:
in short words:

#       How to calculate manual response to send into Authorization header
#       HA1=MD5(username:realm:password)
#       HA2=MD5(method:digestURI)
#       response=MD5(HA1:nonce:HA2)


route[auth] {
            if (!is_present_hf("Authorization")) return;

# <         converts string with ',' to string with ';' 

            $var(raw_auth) = $hdr(Authorization);
            $var(reg_input)=$var(raw_auth);
            xlog("$var(reg_input) [$ci]");
            $var(reg) = "/,/;/g";
            $var(auth) = $(var(reg_input){re.subst,$var(reg)});
            $var(reg) = "/Digest //g";
            $var(auth) = $(var(auth){re.subst,$var(reg)});
            xlog("$var(auth) [$ci]");
# >

            $var(cl_user)     = $(var(auth){param.value,username});
            $var(cl_realm)    = $(var(auth){param.value,realm});
            $var(cl_uri)      = $(var(auth){param.value,uri});
            $var(cl_nonce)    = $(var(auth){param.value,nonce});
            $var(cl_response) = $(var(auth){param.value,response});

#ask asterisk DB for secret
            avp_db_query("SELECT secret FROM ars_sip  WHERE username='$fU'",
                        "$avp(secret)",1);

       if ($avp(secret) == NULL)
            exit;

#       xlog("CL_CREDENTIALS: $var(cl_user) , $var(cl_realm) , $avp(secret)  [$ci]");
        $var(ha1) = $var(cl_user) + ":"+$var(cl_realm)+":" + $avp(secret);

#       xlog("CL_CREDENTIALS: REGISTER:$var(cl_uri) [$ci]");
        $var(ha2) = "REGISTER:"+ $var(cl_uri) ;
        $var(response) = $(var(ha1){s.md5}) + ":" + $var(cl_nonce)+ ":" + $(var(ha2){s.md5});

        $var(response_md5) = $(var(response){s.md5});


        xlog("my $var(response_md5) client response is $var(cl_response)");
        if ($var(response_md5) != $var(cl_response)) 
               exit;

##############

}

24.05.2022

Update opensips 3.2.2 -> 3.2.6 centos 7

процедура такая получилась:
1. удаляем новую Libmicrohttpd
2. обновляем Opensips и ставим http и prometheus модули со старой либой
2.1 копируем модули от нового в tmp
3. Удаляем старую либу(она удаляется с модулями http и prometheus)
4. ставим новую либу Libmicrohttpd
5. копируем модули httpd и prometheus из tmp в папку с Opensips lib
6. делаем копию файла libmicrohttpd.12 -> libmicrohttpd.10
7. после этого можно перезапускать Opensips

command line script:

yum remove libmicrohttpd -y
yum update -y
yum install opensips-http-modules opensips-prometheus-module -y
#copy httpd.so, prometheus.so, mi_http.so > /tmp
yum remove libmicrohttpd -y
yum install libmicrohttpd-0.9.59
#copy all files from /tmp to /usr/lib64/opensips/modules
cp /usr/lib64/libmicrohttpd.so.12 /usr/lib64/libmicrohttpd.so.10
systemctl daemon-reload
setcap CAP_NET_BIND_SERVICE=+eip /usr/sbin/opensips
systemctl restart opensips

30.01.2022

TLS

Немного о том, как настраивать tls_mgm.

TLS domain это обозначение настроек, которое никак не связано с доменами в SIP заголовках. Оно используется для того, что дать opensips возможность определить какие сертификаты использовать для входящих\исходящих соединений. Какие сертификаты (читай: какой TLSdomain) использовать при входящем звонке, Opensips определяет по SIP domain в (SNI) записи в сертификате присылаемом от звонящего, либо по сокету на который пришел запрос на соединение(звонок).

в качестве примера настроек можно выставить
tlsdomain: my_srv_domain,
ip match: “*”,
SIP domain: “*”
certificate: cert1.pem
private ket: privkey1.pem
остальные параметры по-умолчанию.
Таким образом все входящие соединения по TLS будут обработаны этими настройками. (cert1.pem и privkey1.pem это файлы полученные на unix системе certbot приложением).

Для исходящих соединений opensips будет смотреть через какой сокет отправляется звонок. Также можно выбрать TLSdomain через переменную в скрипте ().

28.01.2022

Opensips. MI. Json. Zabbix.

Opensips 3.2 have beautiful statistics module. For example you may get Data about average count of incoming sip messages directly from MI interface. Also you can output it on Zabbix graph.

  1. Enable mi_http module, add into opensips.conf:
    loadmodule “httpd.so”
    loadmodule “mi_http.so”
    modparam(“mi_http”, “root”, “mi”)
  2. Load statistics module and define statistics profiles and add update_stat_series() functions to script, check for example here.

so, now you be able to ask system for stats though MI interface, for example:

opensips-cli -x mi get_statistics all

internally opensips-cli will ask opensips through http://127.0.0.1:8888/mi with POST request with json body:

#example 1 for statistics...
{ 
  "jsonrpc":"2.0",
  "id":1,
  "method":"get_statistics",
  "params":[
             ["avg_1m:",
              "shmem:",
         metri     "usrloc:"]
          ]
}
#example 2 for ratelimit data...
{
  "jsonrpc":"2.0",
  "id":1,
  "method":"rl_list",
  "params": []
}

You will get result in Json format too.
In our case i just counting how many INVITE,REGISTER and CANCELS initial requests caming to my opensips per 1 minute.

#in opensips.conf: 
....
modparam("statistics", "stat_series_profile", "avg_1m: algorithm=accumulate")
....
route { 
      route(custom_stat);
....
}
route[custom_stat] {
                    # Ignore indialog requests
                    if (has_totag())
                        return ;

                        update_stat_series("avg_1m", "$si", 1);
                        update_stat_series("avg_1m", "$rm", 1);
                        update_stat_series("avg_1m", "$socket_in(proto)|$rm|$si", 1);
}

ZABBIX

  1. Create item like HTTP agent
  2. Use (example 1) inside body of POST request, Set JSON type for request and “convert to JSON”
  3. Add preprocessing JSONPath and “$.body.result” see here for more greatfull examples of how to interpret json answers.
  4. next step will be getting exactly params you want to monitor: create another item, but set it as “Depended” on item you have created previously.
  5. Add preprocessing like this : JSON Path and “$.Pipes[?(@.id == “total_INVITE”)].counter” it will show counter value from example 4 Json answer.
{
    "Pipes": [
        {
            "id": "xxx.xxx.xxx.xxx",
            "algorithm": "TAILDROP",
            "limit": 30,
            "counter": 0
        },
        {
            "id": "total_INVITE",
            "algorithm": "TAILDROP",
            "limit": 150,
            "counter": 0
        }
    ],
    "drop_rate": 1150
}

26.01.2022

Opensips-cli. Json. jq.

You know that opensips -x mi dlg_list will produce a lot of JSON output, what if i want to get only dialogs with state = 4 ?
There are beautiful tool like “jq” present in unix. (documentation)

For example output from command “opensips-cli -x mi profile_get_size profile=calls”:

{
    "Profile": {
        "name": "calls",
        "value": null,
        "count": 15,
        "shared": "no",
        "replicated": "no"
    }
}

if i want to get only count number, i can use that:

opensips-cli -x mi profile_get_size profile=calls | jq '.Profile.count' 

And output will be

15

It may be usefull for example when you are using zabbix monitoring. Some useful commands:

//this will output count of dialogs in state of 4 (established)
opensips-cli -x mi dlg_list | jq '.Dialogs[] | select(.state == 4) | .state' | wc -l

//this will count show dialogs have "from = anyfrom@domain.com" and in starting state
opensips-cli -x mi dlg_list | jq '.Dialogs[] | select(.from_uri == "sip:anyfrom@domain.com") | select(.state < 4) | .state' | wc -l

//if you remove "| wc -l" you will see full JSON info about dialogs you requested
//so you can take info about dialogs you want with easy way.

For regexp and tring matches (like i want to see only linphones) you may use this construction:

opensips-cli -x mi ul_dump | jq ‘.Domains[].AORs[].Contacts[] | select(.”User-agent”|test(“Linphone”))’

25.01.2022

Register here to leave comments or asks something

Hey, colleagues, glad to say i am open registration here so you can leave comments.

Всем, привет, на открыл регистрацию здесь – можете оставлять комменты к постам.

17.01.2022

Opensips 3.2, Homer 7

Advantages of using Opensips + Homer is possibility to see webrtc\tls traffic

There is how to set simplest configuration on opensips side and Homer side. Homer 7 instruction for Debian 10.

OPENSIPS:

socket=hep_udp:ens5:9000 
socket=hep_tcp:ens5:9000
...
loadmodule "proto_hep.so"
loadmodule "tracer.so"

modparam("proto_hep", "hep_capture_id", 5002)
modparam("proto_hep", "hep_id",  "[hid]homer_ip:9060; transport=tcp; version=3")
modparam("tracer", "trace_id", "[tid]uri=hep:hid")

####### Routing Logic ########

# main request routing logic

route{

        xlog("INCOME $rm TO: $tu [$ci]");
        trace("tid");
...

HOMER 7: CAUTION use only on vanilla debian due to it will replace pg_hba.conf (old one will ba saved)

apt install curl postgresql mc -y
curl -s https://packagecloud.io/install/repositories/qxip/sipcapture/script.deb.sh | sudo bash
apt install heplify-server homer-app -y
cp /etc/postgresql/11/main/pg_hba.conf /etc/postgresql/11/main/pg_hba.conf.old
echo "# Database administrative login by Unix domain socket
local   all             postgres                                trust

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust

# IPv4 local connections:
host    all             all             127.0.0.1/32            trust

# IPv6 local connections:
host    all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     peer
host    replication     all             127.0.0.1/32            md5
host    replication     all             ::1/128                 md5
" > /etc/postgresql/11/main/pg_hba.conf

systemctl restart postgresql

homer-app -initialize_db
homer-app -create-table-db-config
homer-app -populate-table-db-config
homer-app -upgrade-table-db-config
homer-app -update-ui-user=admin -update-ui-password=mypassword

systemctl restart homer-app

# Set into /etc/heplify-server.toml
# HEPTCPAddr            = "0.0.0.0:9060"
# HEPTLSAddr            = "0.0.0.0:9061"
#

systemctl restart heplify-server

After this you may to connect to your external_ip:9080 port and use admin\mypassword