2. Installation and configuration of the Smarty server¶

2.1. Where to download the installation packages¶

Installation packages are distributed via FTP, access to which is granted for the duration of the contract.

2.2. Installation on Linux Debian¶

All the Smarty modules are shipped as an installable deb packages, and are installable with dpkg utility. smarty-base is an integral module.

Needed packages are nginx web server, uwsgi, python 2.7, and several others.

2.2.1. Installing dependencies¶

Installing via apt-get:

sudo apt-get install nginx git python-dev python2.7 libmysqlclient-dev libtiff5-dev libjpeg62-turbo-dev zlib1g-dev
libfreetype6-dev liblcms2-dev libwebp-dev tcl8.5-dev tk8.5-dev python-tk uwsgi uwsgi-plugin-python redis-server

Installation of the pip utility:

wget https://bootstrap.pypa.io/get-pip.py && python get-pip.py && rm get-pip.py

2.2.2. Installing Smarty and modules¶

Installing via dpkg:

dpkg -i smarty*.deb
dpkg -i python2.7-jsonrpctcp*.deb

After installing Smarty pakages, you need to install the python libraries via pip:

sudo pip install -r /usr/share/nginx/html/microimpuls/smarty/requirements.txt

After installation of the smarty-base package it is being created a configuration file for nginx in /etc/nginx/sites-available/smarty. Defaults to the domain smarty.example.com and listening to port 80 and 8180. Change this domain to yours.

Smarty files are located in /usr/share/nginx/html/microimpuls/smarty.

2.2.3. Installing the database schema¶

Initial installation of the database schema is performed with the command:

smarty_manage migrate --settings=settings.<settings filename>

<settings filename> - the file name of the Smarty settings, in which must be set the connection settings to the database (see Description of the main parameters of).

Note

Additional materials:

the Specifics of installing and configuring Smarty connection to Oracle database

2.2.4. Creating user administrator¶

Create user with service admin rights with the command:

smarty_manage createsuperuser --settings=settings.<settings filename>

<settings filename> - the file name of the Smarty settings, in which must be set the connection settings to the database (see Description of the main parameters of).

2.2.5. Creating system objects in the database and examples of operator settings¶

To create system objects Smarty in the database, as well as example settings, run the command:

smarty_manage setup_initial_data --settings=settings.<settings filename>

<settings filename> - the file name of the Smarty settings, in which must be set the connection settings to the database (see Description of the main parameters of).

Note

You can skip creating the sample operator settings, adding the flag --no-sample-data

2.2.6. Клонирование Client со всеми настройками оператора¶

В случае необходимости создания нового объекта Client и копирования всех данных можно использовать команду клонирования:

smarty_manage clone_client --src_client_id=<source client id> --settings=settings.<settings filename>

<source client id> - ID объекта Client, который нужно склонировать в новый Client.
<settings filename> - the file name of the Smarty settings, in which must be set the connection settings to the database (see Description of the main parameters of).

Note

Можно перенести также все телеканалы, фильмы, радиостанции, камеры и сервисы, используя соответствующие опции --clone-channels --clone-video --clone-radio --clone-cameras --clone-apps

2.2.7. Создание пользователя или восстановление пароля¶

В Smarty возможно создание или восстановление пользователя через команду create_user:

smarty_manage create_user --settings=settings.<settings filename> --username=new_user --password=new_password --is_admin=True --client_id=1 --is_superuser=True

Параметры:

–username - имя пользователя, обязательный.
–password - пароль, обязательный.
–is_admin - True или False, если True, то создаваемому пользователю будет доступна служебная часть сайта, по умолчанию False.
–client_id - ID клиента, к которому будет привязан создаваемы пользователь.
–is_superuser - True или False, если True, то будет создаен суперпользователь, по умолчанию False.
–monitoring_user - True или False, если True, то пользователь будет являться оператором мониторинга устройств, по умолчанию False.
–reset_password - если True, то в случае, если указанный username уже используется, то вместо создания нового будет изменён пароль у старого пользователя; по умолчанию False.

2.2.8. Создание Client¶

В Smarty возможно создание оператора (Client) через команду create_client:

smarty_manage create_client --settings=settings.<settings filename> --name=Client_name --api_key=api_key --domain_prefix=domain_prefix --email=email@example.com

Параметры:

–name - название оператора, обязательный.
–api_key - TVMW API Key, обязательный.
–domain_prefix - префикс домена оператора, обязательный.
–email - email оператора, обязательный.

2.3. Installation on Linux CentOS¶

Support for CentOS is experimental - correct operation of all functions of Smarty is not guaranteed. The setup script for fabric and the configuration examples for CentOS can be found here: https://github.com/microimpuls/smarty-centos

2.4. Smarty Configuration¶

2.4.1. The configuration file Smarty¶

After the initial installation of the base configuration file Smarty is located at the address /etc/microimpuls/smarty/base.py (link to /usr/share/nginx/html/microimpuls/smarty/settings/base.py).

The main configuration file used for production-mode - /etc/microimpuls/smarty/prod.py. This file (or other configuration used) must specify the symlink in /usr/share/nginx/html/microimpuls/smarty/settings/<setings name>.py. It should be set Smarty, because a basic configuration file can be overwritten after updates.

Configuration is made by assigning values to variables in Python.

2.4.1.1. Maintaining multiple Smarty instances on a single server¶

For convenience, the configuration and placement on a single server, multiple instances of Smarty instead of using the settings file prod.py create your own file with a short symbolic name that matches the service name, for example myiptv.py.

This name is then also recommended to use as a prefix or suffix in the names of configuration files nginx, uwsgi, the names of the folders for logs, pid files, etc.

2.4.2. Description of the configuration parameters to Smarty¶

Актуальная документация: https://microimpuls.com/docs/smarty/configuring-and-management/smarty-config https://microimpuls.com/docs/smarty/configuring-and-management/logging

2.4.3. Adding a license key server, Smarty¶

Each instance of Smarty is tied to the hardware and software configuration server license key, which can be limited by time steps and the maximum number of configured Client ID (see Multiprovider).

The license key is configured in the configuration file in the following variables:

SMARTY_KEY = 'XXXXXXX-XXXXXXX-XXXXXXX-XXXXXXX'
SMARTY_MAX_CLIENTS = 2
SMARTY_AVAILABLE_UNTIL = 'dd.mm.yyyy'

To obtain the key you must contact your Manager on the contract.

2.4.4. Setting up caching¶

For caching server is used Redis - is a required component of the system. It requires Redis >= 2.6.

The default configuration assumes a local installation of Redis server on the same server with Smarty, but if necessary, they can be divided. To change connection settings for Redis configuration is required Smarty to register an array CACHES as follows:

CACHES = {
    "default": {
        "BACKEND": "core.cache.backends.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
        }
    }
}

In the configuration file Redis /etc/redis/redis.conf I need to write:

stop-writes-on-bgsave-error no

Для оптимизации дисковых операций и увеличения общей производительности на сервере Redis рекомендуем также указать опцию, запрещающую сохранение на диск текущих данных сервера:

save ""

При ее указании все данные во время работы Redis будут храниться ТОЛЬКО в RAM сервера. В случае перезапуска сервера Redis рекомендуем запустить менеджмент команду flushall для заполнения Redis данными, используемыми Smarty.

Для вступления изменений конфигурации в силу требуется перезагрузить Redis и uwsgi.

Operation is also supported in cluster mode with a group of Redis servers configuration example:

CACHES = {
    "default": {
        "BACKEND": "core.cache.backends.RedisCache",
        "LOCATION": "redis://192.168.33.11:7000/0", # не используется, но необходимо
        "OPTIONS": {
            "REDIS_CLIENT_CLASS": "rediscluster.client.StrictRedisCluster",
            "CONNECTION_POOL_CLASS": "core.cache.cluster_connection.ClusterConnectionPool",
            "CONNECTION_POOL_KWARGS": {
                "startup_nodes": [
                    # masters
                    {"host": "192.168.33.11", "port": "7000"},
                    {"host": "192.168.33.12", "port": "7000"},
                    {"host": "192.168.33.13", "port": "7000"},
                ]
            }
        }
    }
}

2.4.5. Configuration of the module geolocation¶

Supports multiple locators based on IP addresses of the subscriber, working with different sources of geo-data. In service-admin for custom Client ID, you must install the locator used, the most suitable for the operator and its region of service provision.

If you have to change the locator database of countries and cities have already been filled, it is recommended to clean it.

All locators require the creation/updating its database. The database can be in the form of SQL tables, or binary data (or both).

2.4.5.1. Locator django-geoip (ipgeobase)¶

Is a wrapper around the https://django-geoip.readthedocs.org/en/latest/

The command to update the database:

$ smarty_manage geoip_update --settings=settings.<settings name>

The creation of countries and cities based on the data of django-geoip (only works if there is no country and city):

$ smarty_manage sync_geo_geoip --settings=settings.<settings name>

2.4.5.2. Locator ip2location¶

Database update:

$ smarty_manage update_ip2location --settings=settings.<settings name>

This command downloads a binary database to determine the location and CSV database to create a directory of cities and countries.

The creation of countries and cities based on data from ip2location (only works if there is no country and city):

$ smarty_manage sync_geo_ip2location --settings=settings.<settings name>

After you select the locator and synchronization mechanism geolocation is ready to use. The availability of various Middleware services (TV, movies, streaming services, options, etc.) is determined by the tariff packages (see the possibility of billing), in the settings where you can specify the country and city in which they operate.

2.4.10. Настройка модуля отправки SMS¶

Актуальная документация: https://micro.im/docs/smarty/extra-services-integration/интеграция-с-sms-шлюзами

2.4.11. Подключение системы мониторинга ошибок Sentry¶

To connect Sentry need in the configuration file to add Smarty in INSTALLED_APPS module raven.contrib.django.raven_compat and set the connection parameters:

RAVEN_CONFIG = {
    'dsn': 'http://<LOGIN>:<PASS>@<SENTRY HOST>/<PROJECT>',
}

The connection string can be obtained from the project settings in Sentry.

2.4.12. Настройка nginx и uwsgi¶

A sample configuration file for the nginx is in the file /etc/nginx/sites-available/smarty.

Configuration for uwsgi is in the file /etc/uwsgi/apps-available/smarty and /etc/microimpuls/smarty/uwsgi/smarty.uwsgi on it (or another used the config) must specify the symlink in /usr/share/nginx/html/microimpuls/smarty/<uwsgi settings name>.uwsgi.

2.4.13. Настройка мультиязычности контента в Smarty¶

Smarty allows you to save database content of names of localizable fields in different languages - for example, the names of channels, movies, categories, genres, EPG etc. to activate this mechanism, you must add a configuration file option SMARTY_ADDITIONAL_LANGUAGES list of required languages (not more than 5 additional to the main languages), and specify the primary language. The language names must match the locale name in the user application, the default is to use the two-letter names.

SMARTY_DEFAULT_LANGUAGE str: The name of the main language. Default EN.
SMARTY_ADDITIONAL_LANGUAGES list: A list of additional languages, specified in square brackets with indication of values separated by commas, for example: [ 'en', 'fr', 'de', 'es', 'pt' ] default empty.

After setting up of multilingual settings and reboot uwsgi in the admin panel Smarty in the form of localizable fields will have the opportunity to indicate the name in additional languages.

For the server to Smarty in response to the request the API returned the value in the desired language, you must additionally pass a parameter lang. Read more in the documentation TVMiddleware API.

2.5. System team Smarty and configure crontab¶

Note

Attention! Some commands of the scheduler are required for the operation of the service.

2.5.2. Импорт EpgChannel¶

Team:

smarty_manage epg_channel_import --settings=settings.<settings name>

Данная команда поможет загрузить все каналы или обновить иконки из определенного источника. Для запуска обязательно необходимо указать epg_source_id или --epg_source_name.

Если команда вызывается для загрузки иконок или загрузки всех каналов из EpgChannelSource, то к загруженным иконкам по возможности будут созданы более маленькие копии следующих размеров: 500x500, 120x91 и 40x40.

Для источника обязательно должен существовать EpgChannelSource с указанием маски URL источника каналов. Для источников, у которых нет общего списка необходимо указывать epg_channel_id или channel_id.

Аргументы для запуска:

`--epg_source_id`
	Идентификатор EpgSource, для которого необходимо произвести импортирование каналов.
`--epg_source_name`
	Имя EpgSource, для которого необходимо произвести импортирование каналов.
`--epg_channel_id`
	Идентификатор EpgChannel, для которого необходимо произвести импортирование.
`--channel_id`	Идентификатор Channel, для которого необходимо произвести импортирование.
`--reimport_icons`
	Если указан этот аргумент, то для всех импортированных каналов будет произведено обновление иконок.
`--force_import`	Загрузка всех каналов из источника. Если указан данный аргумент то все аргументы кроме `epg_source_id` и `epg_source_name` будут проигнорированы.
`--force_parser_handling`
	Принудительно разрешает использовать передачу управления парсеру (равносильно `TVMW_EPG_IMPORT_ALLOW_PARSER_HANDLING=True`)
`--force_disable_parser_handling`
	Принудительно запрещает использовать передачу управления парсеру (равносильно `TVMW_EPG_IMPORT_ALLOW_PARSER_HANDLING=False`)
`--fix_duplicates`
	Удаляет дубликаты телеканалов с одним и тем же внешним идентификатором в рамках одного источника EPG.
`--verbose`	Включает подробный вывод информации о загружаемых из источника иконках.

Использование опций –force_parser_handling и –force_disable_parser_handling приоритетнее параметра TVMW_EPG_IMPORT_ALLOW_PARSER_HANDLING.

Пример команды для повторного импортирования иконок для одного канала:

smarty_manage epg_channel_import –epg_source_id=1 –epg_channel_id=100 –reimport_icons –settings=settings.<settings name>

2.5.14. Команда кэширования существующих иконок¶

Team:

smarty_manage recache_icons --settings=settings.<settings name>

Вызывается в случае отсутствия информации о существующих иконках.

Команда проверяет и сохраняет в кэше существование иконок для всех EpgChannel по размерам, указанным в SMARTY_DEFAULT_ICON_SIZE и SMARTY_DEFAULT_ICON_SIZES.

2.5.19. Пример настройки crontab¶

For example:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
*/1 * * * *         python /usr/share/nginx/html/microimpuls/smarty/manage.py cache_channel_list --settings=settings.prod
0 5,9,13 * * *      python /usr/share/nginx/html/microimpuls/smarty/manage.py epg_import --settings=settings.prod
0 3 * * *           python /usr/share/nginx/html/microimpuls/smarty/manage.py clean_old_messages --days_count 3 --settings=settings.prod

2.5.20. Команда генерации видео для архивных записей¶

Team:

smarty_manage make_vodpvr --client_id=<client_id> --settings=settings.<settings name>

Выполняет создание видео для каждой программы, если для нее существует канал ведущий архивную запись. Отличается от контента

2.6. Start, restart and stop Smarty¶

For the process control application server uses uwsgi init script /etc/init.d/uwsgi:

$ /etc/init.d/uwsgi
Usage: /etc/init.d/uwsgi {start|stop|status|restart|reload|force-reload}

All commands operate on all running uwsgi instances.

The logs are stored by default in /var/log/uwsgi/, /var/log/nginx/ and /var/log/microimpuls/.

2.7. Install updates Smarty¶

Note

Before installing the service packs, please make a backup of the configuration files, Smarty, and database dump.

Updates are installed by a team dpkg:

dpkg -i smarty*.deb

After you install the update you need to install the new required python libraries using pip:

sudo pip install -r /usr/share/nginx/html/microimpuls/smarty/requirements.txt --ignore-installed

Migration of the database schema with the command:

python /usr/share/nginx/html/microimpuls/smarty/manage.py migrate --settings=settings.<settings filename>

Если в процессе установки пакета будет предложено заменить файл base.py - необходимо ответить Y (заменить файл).

After installing all the upgrade and migration of database schemas, you must restart the application server, uwsgi, complete all commands epg_import and cache_channel_list (via a call to kill), and then run a command to update the cache:

python /usr/share/nginx/html/microimpuls/smarty/manage.py flushall --settings=settings.<settings filename>

Note

If you do not run the command to update the cache flushall, then the cache may be the data from the old structure, which can lead to unpredictable errors in the applications.

2.7.1. Troubleshoot conflict migrations¶

During the migration of the database schema error may occur conflict migration: To fix them run 'python manage.py makemigrations --merge'. No need to make a team makemigrations. Error may occur in case of violation of correct procedure when installing or updating a system. To resolve this error, you must perform the following steps:

To remove the contents of the folder /tvmiddleware/migrations
To reinstall service pack smarty, which was installed at the time when the error occurred. Temporarily remove from the folder /tvmiddleware/migrations all available new migrations (creation date, which is newer than the date of the last successful update smarty).
Clear table django_migrations database in smarty.

Run the command:

$ python manage.py migrate --fake --settings=settings.<settings filename>

Copy back the migration, is temporarily removed in step 2. Re-run the migration command:
```
$ python manage.py migrate --settings=settings.<settings filename>
```

2.8. Scalability and fault tolerance¶

2.8.1. Available scaling and fault tolerance¶

Подробная информация о кластерных конфигурациях Smarty в новой базе знаний по продукту на сайте https://microimpuls.com/docs/smarty/scaling-and-redundancy.

2.8.2. Configure a connection to a DBMS using replication¶

Актуальная документация по настройке подключения к SQL-кластеру: https://microimpuls.com/docs/smarty/scaling-and-redundancy/smarty-sql-cluster-connection

2.8.3. Configure the logging of query statistics to statsd¶

statsd - server aggregate statistics: https://github.com/etsy/statsd.

Smarty allows you to upload a statsd statistics API requests (number of requests, response time, number of executed SQL queries, the response time of the DBMS). To do this, you must add the configuration file Smarty the parameters listed below:

MIDDLEWARE_CLASSES += (
    'core.middleware.StatsMiddleware',
)

STATSD_HOST = 'X.X.X.X'
STATSD_PORT = '8125'

Where:

STATSD_HOST str: The IP address of the statsd server to upload data for statistics and monitoring of servers with Smarty.
STATSD_PORT int: The port of the statsd server to upload data for statistics and monitoring of servers with Smarty.
STATSD_PREFIX str: The prefix that will be added (if used) to the key parameters that are passed to statsd.

Note

Внимание! Необходимо обеспечить доступность сервера statsd и правильность настроек подключения, в противном случае подключенная `core.middleware.StatsMiddleware` и отсутствие соединения со statsd может приводить к чрезмерному потреблению оперативной памяти.

2. Installation and configuration of the Smarty server¶

2.1. Where to download the installation packages¶

2.2. Installation on Linux Debian¶

2.2.1. Installing dependencies¶

2.2.2. Installing Smarty and modules¶

2.2.3. Installing the database schema¶

2.2.4. Creating user administrator¶

2.2.5. Creating system objects in the database and examples of operator settings¶

2.2.6. Клонирование Client со всеми настройками оператора¶

2.2.7. Создание пользователя или восстановление пароля¶

2.2.8. Создание Client¶

2.3. Installation on Linux CentOS¶

2.4. Smarty Configuration¶

2.4.1. The configuration file Smarty¶

2.4.1.1. Maintaining multiple Smarty instances on a single server¶

2.4.2. Description of the configuration parameters to Smarty¶

2.4.3. Adding a license key server, Smarty¶

2.4.4. Setting up caching¶

2.4.5. Configuration of the module geolocation¶

2.4.5.1. Locator django-geoip (ipgeobase)¶

2.4.5.2. Locator ip2location¶

2.4.6. Setting of the monitoring module of the video streams¶

2.4.7. Configuration of the module statistics and reports¶

2.4.8. Настройка модуля сбора статистики по абонентам¶

2.4.9. Настройка модуля мониторинга устройств¶

2.4.10. Настройка модуля отправки SMS¶

2.4.11. Подключение системы мониторинга ошибок Sentry¶

2.4.12. Настройка nginx и uwsgi¶

2.4.13. Настройка мультиязычности контента в Smarty¶

2.5. System team Smarty and configure crontab¶

2.5.2. Импорт EpgChannel¶

2.5.14. Команда кэширования существующих иконок¶

2.5.19. Пример настройки crontab¶

2.5.20. Команда генерации видео для архивных записей¶

2.6. Start, restart and stop Smarty¶

2.7. Install updates Smarty¶

2.7.1. Troubleshoot conflict migrations¶

2.8. Scalability and fault tolerance¶

2.8.1. Available scaling and fault tolerance¶

2.8.2. Configure a connection to a DBMS using replication¶

2.8.3. Configure the logging of query statistics to statsd¶