Master Service Issues

The Master Service

One of the three critical services that run on your hosted server, the "Master" service is in constant communication with your BATMs. All BATM transactions & events use this channel.

The Master service exposes ports 7741 and 7743. Your BATMs talk to CAS via port 7741, and CAS exposes extensions and some internal interfaces via port 7743.

If the Master service is disrupted, your BATMs will all fail to connect. They will all be offline.

Symptom:

The underlying cause of this type problem may arise in any of five areas:

Open

This article addresses common CAS issues.

Confirm the symptom.

There’s no reason to try the “solutions” in this article if you haven’t confirmed the problem/symptom.

First, identify if the Master service is running.

1. Login to your CAS server console via SSH.

2. From your server's command line interface (CLI), type:

• sudo /batm/batm-manage info

Investigate the Master log:

You should still be able to access the CAS GUI/Admin. You can easily view the Master log from there:        

• Navigate to: Log Viewer/Master Log

• Click on "GO TO END OF FILE".

• Examine the log for errors which might explain the problem (“FATAL”).

• The Master log contains communication notes between your server and BATMs and contains a great deal of troubleshooting information.

Start the Master service using the “fg” option.

The Master service can be started in verbose mode using this special command:

• The Master service will then attempt to start - but it will display ALL messages onscreen - successes AND failures. Not all failures are unexpected or unwanted; they are merely reporting tools and the information is used for complex debugging.

• Errors reported towards the end are the most relevant.

If the Master service is started without the “fg" switch (using “start"), then the Master log will be concatenated to any previous Master log, and can be examined live by typing:

Try these possible solutions:

Ubuntu should be updated regularly, and it will need to be restarted occasionally. This is easy to overlook. If the Master service is suddenly problematic, update Ubuntu and reboot your server.

• Should report at least 5gb free for the Master service to run reliably.

• If the CPU or memory of any individual process exceeds 50%, you may need to increase your system specifications.

• Usage surges (of either type) may result in random intermittent server reboots.

• Memory depletion may result in the unpredictable termination of processes (killed).

The Master service may shut down if it is unable to communicate with your rate provider for a substantial period of time. This is for your protection - to prevent sales & purchases that may be considerably askew from actual current prices. Always check that your rate provider is reliable and functioning. This shutdown may occur intermittently!

This measure may reduce Out Of Memory crashes. This unusual measure requires you to edit the batm-manage file: /batm/batm-manage

Locate the line containing:

Add the following (as instructed by Support) “-Xmx8g" in the following location:

• The setting may change depending upon your specific needs, and

• the example provided is for CAS hosts with 16GB+ of RAM.

Digital Ocean droplets do not have a swap file (by default).

A swap file is used by Linux to move memory to the file system when cramped. Linux (and more especially MySQL) uses A LOT or memory, and in fact MySQL will use all available permitted memory. Even if you increase your RAM to 256GB, MySQL may use almost all of it - resulting in continued occasional crashes. The solution is a swap file.

Check to see if you have a swap file in place with:

Create a swap file if none exists (if you see this):

Open

To create a 2GB swap file (any size > 1GB on a 8GB system should suffice):

To make the change permanent, edit the /etc/fstab file and append /swapfile swap swap sw 0 0 to it:

• Press Ctrl+X and “Y” to save and exit the editor.

The swap file should already be active, but verify with:

• The swap file should now show a number greater than zero.

• The swap file should still exist even after rebooting your system.

• hardware failure,

• bandwidth limitations (each BATM may consume up to 4gb/mo),

• attacks by malicious actors overloading your server,

• JAVA version mismatch. Installing extra software on your server may cause your server's JAVA be switched to a different, incompatible version. YOUR SERVER SHOULD BE DEDICATED TO CAS!

These problems are outside the scope of General Bytes support. Please contact your hosting provider or a third-party trusted vendor to address server issues and/or required upgrades. Watch your server usage graphs (if available). Use the "top" utility to watch processor load. It should stay well below 50% on a normal system.

• https://generalbytes.atlassian.net/wiki/spaces/ESD/pages/934609035and

• https://generalbytes.atlassian.net/l/cp/fqN1R1GM for further investigation by Support.

Related articles