How to Troubleshoot 596 Error Caused by Memcache

The 596 Service not found error may be caused by memcache and you may see the following errors in proxy log:

[2017-03-21T19:16:42+00:00] WARN  [Memcached IO over {MemcachedConnection to ml-mem/172.19.0.2:11214}] n.spy.memcached.MemcachedConnection  - Closing, and reopening {QA sa=ml-mem/172.19.0.2:11214, #Rops=0, #Wops=2, #iq=0, topRop=null, topWop=Cmd: get Keys: ENDPOINTS_digital-api.biogen.comExp: 0, toWrite=0, interested=0}, attempt 2.
 
 
[2017-03-21T19:16:44+00:00] WARN  [proxy-server-71] c.m.p.i.m.MemcachedClientImpl  - Operation timed out; retrying...
 
 
net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: ml-mem/172.19.0.2:11214

First check if whether the memcached is running:

docker exec -it ml-mem ps -ef

Then, look for the memcached process. If not running, get in the ml-mem container to start it and see whether there's any error:

docker exec -it ml-mem /bin/bash
then
 
 
service memcached start

If it failed to start because of running out of file limit, following the instructions in the section How to Change Ulimits on Containers to fix it.