Menu

Upgrade Steps (IV 4.X or IV 5.X)

Intelligent Voice
  • Updated
Follow

App Node Upgrade

  1. Stop all workers
  2. Stop Gearman

  3. Backup fail queue in database

    mysqldump 
    mysqldump gearman gearman_queue --where="function_name = 'cracker_fail'" >> /tmp/backup.005/cracker_fail.sql
mysqldump gearman gearman_queue --where="function_name = 'tagger_fail'" >> /tmp/backup.005/tagger_fail.sql

  4. Backup App files

    App Node Backup 
    ################################################################################
### app_backup.sh
################################################################################

#!/bin/bash

BACKUP_DIR="/var/backup.00X/"

mkdir -p  ${BACKUP_DIR}
mkdir -p ${BACKUP_DIR}/WEB-INF
mkdir ${BACKUP_DIR}/default

cp -p /opt/jumpto/vrx.properties ${BACKUP_DIR}
cp -pr /opt/jumpto/ssl ${BACKUP_DIR}
cp -p /etc/systemd/system/*.mount ${BACKUP_DIR}

## MySQL
cp -pr /etc/my.cnf.d ${BACKUP_DIR}
cp -p /etc/my.cnf ${BACKUP_DIR}

## Tomcat Files
cp -p /opt/tomcat/webapps/vrxServlet.war ${BACKUP_DIR}
cp -p /opt/tomcat/webapps/vrxServlet/WEB-INF/*.xml ${BACKUP_DIR}/WEB-INF
cp -pr /opt/tomcat/conf ${BACKUP_DIR}
cp -p /opt/tomcat/webapps/vrxServlet.war ${BACKUP_DIR}
cp -p /etc/default/tomcat8 ${BACKUP_DIR}/default

## Old installer config
cp /media/install/iv-web-app-4.11.9.1_rh_7.5-rh/install_standalone/install.cnf ${BACKUP_DIR}

  5. For UAT only, store the output of the following sql commands for restoring database after upgrade

    Save DB Output 
    use obsilon;
select max(id) from 1_mail;
select max(id) from 1_evitem;
select * from model;

  6. Extract new installer

    extract new installer 
    mkdir -p /media/install/iv-web-app-4.11.9.4-rh_7.5/
tar xzvf iv-web-app-4.11.9.4-rh_7.5.tar.gz -C /media/install/iv-web-app-4.11.9.4-rh_7.5/
  7. Edit installer files

    6.1 Update install.cnf
    For UAT, set overwite_existing_data = "Y"
    For Prod, set overwrite_existing_data = "N"

    6.2 Update playbooks/site.yml
    Copy site.yml to site.yml.000 for backup, then open site.yml and comment out all lines except "databases"

  8. Run the installer
    ./install_app.sh
  9. Restart tomcat8

  10. Update vrxServlet.war

    Update vrxServlet.war 
    chmod 644 /media/install/iv-web-app-4.11.9.4-rh_7.5/install_standalone/local_files/tomcat8/files/tomcat/webapps/vrxServlet.war
chown tomcat8:tomcat8 /media/install/iv-web-app-4.11.9.4-rh_7.5/install_standalone/local_files/tomcat8/files/tomcat/webapps/vrxServlet.war
cp -p /media/install/iv-web-app-4.11.9.4-rh_7.5/install_standalone/local_files/tomcat8/files/tomcat/webapps/vrxServlet.war /opt/tomcat/webapps
  11. Restart tomcat8 again and check the j2ws.log
  12. Restore config files (XML files, vrx.properties)

  13. Verify XML files

    Verify XML Files 
    asr-batch-job-scheduler.xml
diar-job-scheduler.xml
import-completed-job-scheduler.xml
import-item-complete-job-scheduler.xml
load-phoneme-job-scheduler.xml
email-2-relativity-job-scheduler.xml
audio-2-relativity-job-scheduler.xml
chat-2-relativity-job-scheduler.xml
check-relativity-job-scheduler.xml
ing-email-eu-2-relativity-job-scheduler.xml
ing-audio-eu-2-relativity-job-scheduler.xml
ing-chat-eu-2-relativity-job-scheduler.xml
ing-check-eu-relativity-job-scheduler.xml
ing-delete-voice-item-job-scheduler.xml
ing-delete-email-item-job-scheduler.xml
chat-import-finsihed-job-scheduler.xml
delete-chat-item-job-scheduler.xml
  14. Start gearman-job-server, start tomcat8

  15. For Prod, APP upgrade COMPLETE
    For UAT, as database is cleaned up, need to create_group and restore the database

    Restore DB 
    use obsilon;
alter_table 1_evitem auto_increment = 1010000;
alter_table 1_mail auto_increment = 1010000;
insert into model (created_by, language_code, sample_rate, lexicon_size, description, version, training_method, model_status_id, ocr_language_id) values ( <stored_output> );

Proc Node Upgrade

  1. Backup Files

    Proc Node Backup 
    ################################################################################
### proc_backup.sh
################################################################################
#!/bin/bash

BACKUP_DIR="/var/backup.00X/"

mkdir -p ${BACKUP_DIR}

cp /etc/systemd/system/*.mount ${BACKUP_DIR}

cp /opt/ReutersChatConnectorService/ticket.db ticket.db.ReutersChatConnectorService
cp /opt/SalesforceChatterConnectorService/ticket.db ticket.db.SalesforceChatterConnectorService

cp /opt/IMAPService/*.pkl ${BACKUP_DIR}

cp /opt/ChatImportWorker/lib/config.py ${BACKUP_DIR}/config.py.ChatImportWorker
cp /opt/CheckRelativityWorker/lib/config.py ${BACKUP_DIR}/config.py.CheckRelativityWorker
cp /opt/DeleteItemWorker/lib/config.py ${BACKUP_DIR}/config.py.DeleteItemWorker
cp /opt/IMAPService/lib/config.py ${BACKUP_DIR}/config.py.IMAPService
cp /opt/ReutersChatConnectorService/lib/config.py ${BACKUP_DIR}/config.py.ReutersChatConnectorService
cp /opt/ReutersChatImportWorker/lib/config.py ${BACKUP_DIR}/config.py.ReutersChatImportWorker
cp /opt/SalesforceChatterConnectorService/lib/config.py ${BACKUP_DIR}/config.py.SalesforceChatterConnectorService
cp /opt/SalesforceChatterImportWorker/lib/config.py ${BACKUP_DIR}/config.py.SalesforceChatterImportWorker

## old installer config
cp /media/install/iv-proc-node-installer-1.9.2.3.1-patched/common-config ${BACKUP_DIR}

cp /opt/IV/kaldi/groupProcess/ASR.conf ${BACKUP_DIR}/

cp -r /opt/jumpto/ssl ${BACKUP_DIR}/


for x in ASRCPUWorker ASRWorker Diar2Worker LMAWorker punctuator TaggerWorker VADWorker; do
    mkdir -p ${BACKUP_DIR}/${x}                           ## optional step
    cp /opt/${x}/*.config ${BACKUP_DIR}/${x}/             ## ASRCPUWorker.config
                                                          ## ASRWorker.config
                                                          ## Diar2Worker.config
                                                          ## LMAWorker.config
                                                          ## punctuator/psrt_mangler.pl.config, punctuator.config
                                                          ## TaggerWorker.config, TaggerWorker.pl.config
                                                          ## VADWorker.config
done

  2. Upload and untar new installer

    extract new installer 
    tar xzvf iv-proc-node-installer-1.9.2.3.4.8.2.tar.gz -C /media/install/

  3. Update new installer config files and scripts

    update config files 
    cd /media/install/iv-proc-node-installer-1.9.2.3.4.8.2
# update hostname and custom users for workers
vi common-config
# remove sphinxsearch and SSLCert in the list of components to install
vi installation_scripts/install_proc.sh
# do not start service when services are updated: install_systemd_service => comment out the line to start service
vi common-functions

  4. Update new installer directory files comparing to backup copies

    update services files 
    diff _ChatImportWorker/chat_import_worker/lib/config.py ${BACKUP_DIR}/config.py.ChatImportWorker
## remove US and APAC in RELATIVITY, leave EU and default (set workspace_ids, import_auth, RELATIVITY_FIELD_TYPE_INFO, url)
## edit WORKER_COUNT
diff _CheckRelativityWorker/check_relativity_worker/lib/config.py ${BACKUP_DIR}/config.py.CheckRelativityWorker
diff _DeleteItemWorker/delete_item_worker/lib/config.py ${BACKUP_DIR}/config.py.DeleteItemWorker
## remove US and APAC in IMAP_ACCOUNTS, leave EU and default (update workspace_ids, import_*_folder)
## update USE_EWS, IMAP_CONNECTION_DELAY, IMAP_CONNECTION_MAX, WORKER_COUNT, IV_GROUP_ID, MONITORED_USERS_FILE
diff _IMAPService/imap_service/lib/config.py ${BACKUP_DIR}/config.py.IMAPService
diff _punctuator/psrt_mangler.pl.config ${BACKUP_DIR}/punctuator/psrt_mangler.pl.config
diff _punctuator/punctuator.config ${BACKUP_DIR}/punctuator/punctuator.config
diff _ReutersChatConnectorService/reuters_chat_connector_service/lib/config.py ${BACKUP_DIR}/config.py.ReutersChatConnectorService
diff _ReutersChatImportWorker/reuters_chat_import_worker/lib/config.py ${BACKUP_DIR}/config.py.ReutersChatImportWorker

  5. Verify app server is ready (no ingestion to queues) and then Run the installer

    run installer 
    cd installation_scripts/
./install_proc.sh

  6. Uninstall old workers

    uninstall old workers 
    cd _CrackerWorker
./uninstall.sh
cd _TaggerWorker
./uninstall.sh
cd _LMAWorker
./uninstall.sh
cd _sphinxsearch
./uninstall.sh
cd _Diar2Worker
./uninstall.sh
cd _CheckRelativityWorker
./uninstall.sh
cd _ChatImportWorker
./uninstall.sh

  7. Put items to a temp queue for testing each worker one-by-one later

    put items to _temp in gearman queue 
    mysql
update gearman.gearman_queue set function_name = 'chat_import_temp' where function_name = 'chat_import';
update gearman.gearman_queue set function_name = 'email_export_temp' where function_name = 'email_export';
update gearman.gearman_queue set function_name = 'check_relativity_temp' where function_name = 'check_relativity';
update gearman.gearman_queue set function_name = 'vad_temp' where function_name = 'vad';

Related articles

Please sign in to leave a comment.

Powered by Zendesk