Common Debugging Scenarios
This section discusses some common problems and general comments about debugging these issues. Note that since we discussed the debug pages and traces, we reference these sections and does not provide detailed explanations on these functions again.
Key or Certificate Log ins are Failing
For Incoming requests: (i.e. Client connects to MFT server)
-
Make sure that the MFT Service (FTP, SSH, and Platform Server) Client Authentication is configured for Certificate or Password, Certificate and Password or Certificate Only. Otherwise, the MFT Server does not request a client certificate.
-
Go to Configuration > System Configuration.
-
Look for the Global FTP/SSH/Platform Server settings.
-
-
Make sure that the client has sent their key or certificate to the MFT admin.
-
Make sure that the MFT Admin has associated the key or certificate with the incoming user ID.
-
Make sure that the SSH client is sending the correct user ID to the MFT server. Look in the
../logs/audit/login-audit-yyyy-mm-dd.log
file for an entry that matches the time of the incoming request -
If necessary, enable log in tracing on the Internet Server. See the section above on log in tracing.
For Outgoing requests: (i.e. MFT client connects to target server)
-
After adding the partner server, go to the Manage page, select the server and retrieve the public key for that server.
-
The server definition defines the system key that is used when communicating to the target system. For SSH, the transfer definition can override the server system key.
-
Make sure that we have sent the public key associated with the system key to the Transfer partner.
-
Make sure that you have defined the used and a dummy password in the server or transfer definition. Some protocols need to know the user associated with the public key.
-
If outgoing requests still get authentication errors, contact the transfer partner and find out the error message they are getting.
-
You can turn on enable tracing for the target SSH server. Depending on the cause of the failure, you might get some additional information on the cause of the failure; however more likely, the authentication exception is displayed.
A Platform Server Client receives error: "This file is not eligible to be transferred. No suitable File Transfer Definition for this alias found"
There are a few possible causes of this failure:
-
The Platform Server client user does not have a transfer definition that matches the requested Virtual Alias. Use the Manage Transfers page to search for a transfer definition that matches the user, Virtual Alias and Upload/Download.
-
The Platform Server Remote File parameter defines the Virtual Alias. For example:
RF:abc/file1.txt
. "abc
" is the Virtual Alias" andfile1.txt
is the file name. -
The Virtual Alias is case-sensitive and must exactly match the directory name in the Platform Server Remote File Name field. Note that MFT removes leading and trailing slashes.
Files are not displayed in the user's download directories
Most likely MFT was unable to contact the target server to a directory list.
-
Look at the Error Events.
-
Go to Diagnostics > Error Events.
-
Looks for an error event that matches the time the error occurred.
An exception is generated when an admin user accesses an admin page
When an error occurs on an admin page, the stack trace for the error is not displayed on the screen. If tracing is enabled, the stack trace must be written to the user trace. If tracing is not enabled, the stack trace must be written to the catalina.out
file in <MFT-Install>/server/logs
.
Generally speaking, TIBCO Technical Support needs the stack trace for this issue. You must also navigate to the Diagnostics page for the server and click Save Server Diagnostics to File.
An SSH client initiates a file transfer that is failing
First you need to find out when the transfer failed. There are a few places where the transfer could fail:
-
The transfer started. You can tell if a transfer started by looking for an audit record. The Audit record should have two fields that gives you more information about the failure:
-
Transfer Status Msg
-
Proxy Status Msg
-
-
If an audit record was written, then the SFTP Client negotiation was successful, the user successfully logged in and the Virtual Alias was found. Most likely the failure was connecting to the target server to upload or download the file.
-
If an audit record was not written, then there are a few possible errors:
-
If the error was an authentication failure, a record must be written to this file:
../logs/audit/login-audit-yyyy-mm-dd.log
. If you see an error in this file, then the SSH negotiation was successful, but authentication failed. -
There was a negotiation error in negotiating the SSH session. The
catalina.out
file might contain a stack trace with the error received. More likely, the SSH client might show the cause of the error, since the SFTP client determines the SSH algorithms actually used. If you cannot determine the cause of the failure, you might need to enable tracing. Since the user has not logged in yet, a user trace does not help. You might need to enable SSH Server Tracing. See the information above about enabling SSH tracing.
-
A Platform Server initiates a file transfer that is failing
Unlike other protocols, the cause of the error is not hidden from other Platform Server clients and servers. A Platform Server audit record is written for each Platform Server transfer attempted. Use the cfinq
utility (UNIX and Windows) or Platform Server panels(z/OS and IBMi) to see the cause of the failure. An Internet Server Audit Record is only written if the transfer actually starts. This means that the following must have occurred:
-
The Platform Server Client was able to connect to the Internet Server port.
-
SSL/TLS negotiation has been successful.
-
Authentication using the credentials (Certificate or password) for the user was successful.
-
The Virtual Alias specified in the remote file name matched a valid transfer definition for that user.
User traces are often helpful in cases like this. You can enable traces for the user ID for the transfer definition. Typically, it is a good idea to enable tracing on the object with the least usage; this is typically the transfer definition. Traces for Platform Server transfers write information about all data sent over the TCP Connection; these traces can get very large and can slow down transfers.
An LDAP Sync is failing
When an LDAP Sync fails, only general information is displayed on the admin console. You might need to look at the catalina.out
and possibly enable tracing to determine the cause of the failure. Here are some things you can do to debug LDAP sync issues:
-
Is the issue just for LDAP Sync? Can LDAP users log in to MFT? If LDAP users can log in to MFT, then the connectivity to LDAP is good; the problem might be just for LDAP Sync.
-
Test the LDAP Authenticator.
Go to Configuration > Authenticators > Manage Authenticators.
Select the Test button next to the authenticator that you want to test. This displays a count of users that would be synced as well as the rights that would be synced.
-
Try to sync one user.
Go to Administration > LDAP Sync.
Click Sync user.
Enter the user ID in format: authenticator-userid
Click Sync.
Only this user is synced.
-
TIBCO Technical Support might request a trace of the LDAP Sync. Enable tracing for the user performing the LDAP Sync. Then perform the LDAP Test or the LDAP Sync and then disable tracing for the user. The user trace should show more information about the LDAP Sync failure.
-
Each LDAP Sync writes information to the following file:
<MFT-Install>/logs/messages/ldap_sync_report_messages-MFT-yyyy-mm-dd.txt
-
TIBCO Technical Support might request this file.
A Transfer Event Alert is not executing
Transfer Alerts are executed when a transfer is about to be written to the Audit database. If no record is written to the Audit database, then no alert is executed. Alerts are executed differently for Internet Server and Platform Server transfers.
Internet Server Transfers: Alerts are executed on the Internet Server where the transfer executed
Platform Server Transfers: Alerts are executed on the Command Center where the Platform Transfer was collected
Before proceeding, you need to determine if the alert was not executed, or if the alert was executed but the alert action failed. You can search the alerts that have been executed.
-
Go to Reports > Alert History > Search Alerts.
All alerts that have been executed today are displayed.
-
Select the entry where the "Alert Description" matches the alert definition. You can then look for the action to see if it executed successfully.
-
If you see a matching Alert History record, the alert did execute. Now you must check the Alert Actions:
-
Click the Alert Audit Id to get the detailed information on the alert.
-
Look for the Alert Action. The action status is displayed for all alert actions executed. If the request failed, the Status field displays the exception that caused the request to fail.
If you do not see a matching alert history record, then the alert did not execute. You must perform the following checks:
-
Verify that the transfer completed and was written to the audits table.
-
Go to Reports > Audits > Search Audits.
-
Set the filter criteria to search for the transfer that should have generated an alert. If no transfer was found, then the transfer was not executed; therefore, no alert was generated.
-
If a transfer audit record was found, compare the transfer audit parameters to the Alert Trigger Criteria. Make sure that all of the Trigger criteria defined matches the audit record. Depending on how the trigger criteria are defined, the trigger comparisons might be case-sensitive.
-
If you still cannot determine why the alert is not written, then TIBCO Technical Support might need you to enable tracing. For an Internet Server transfer, enable tracing for the transfer definition. For Platform Server, you need to enable tracing for the Collector. To do this, enable tracing for user "Collector", run the Platform transfer again, wait for the platform transfer to be collected and then turn off tracing.
A Transfer Non-Event Alert is not working
Transfer non-event Alerts must be configured in two places:
-
Alert Definition: Defines the Trigger criteria and the actions to be executed
-
Scheduler definition: Defines when the job checks if the transfer has executed.
There are a few reasons that a non-event alert is not reporting that transfers have not executed:
-
The Non-Event Transfer Alert scheduler job was not configured or was configured incorrectly.
-
A transfer actually executed that matched the alert trigger criteria. So, the alert is not triggered.
Check the events (Diagnostics > Events > Search Events) to see if the alert executed.
The Collector is not collecting transfers from a Platform Server
When a server is not collecting transfers, there are a few possibilities:
-
If you just enabled collection for this Platform Server, you must restart the collector for this change to take effect.
-
There is no connectivity to the target Platform Server or the target Platform server is not active or the IP name or port is incorrect.
-
The credentials passed to the Platform Server are incorrect.
-
The Platform Server is rejecting the Collection request. For collections to work, the Platform Server needs the following configurations:
-
The Platform Server Node definition for the Command Center must be defined with Command Center Support=Audit or All.
-
The user that is executing the collector request must be a member of the
cfadmin
orcfbrowse
groups.
-
Each time a collection request is made to a Platform Server, Command Center writes a record to this file:
CollectorAudit-audit-MFT-app-YYYY-MM-DD.log
This file displays a status message that includes this information:
-
Start and end date/time of the collection request.
-
The Server name (called Nodename in this file) that was being collected.
-
The Status Message (displays a descriptive message if the request fails).
-
The number of transfers collected(NumCollected) and the number written to the database (NumProcessed).
-
The ID of the last record collected.
The records in this file should tell you why the Collection is not working.
A scheduler request is not executing properly
First check if the scheduler job did execute.
Go to Diagnostics > Events > Search Events.
One record is written to the Events database for each scheduler job that is executed. Use the Search Criteria to filter the events that are returned. If you see the job, click the Event Id to get the detailed record. The detail record is displayed. The detail record shows the following information:
-
The start and end times for the job
-
The Job Type (Event Type)
-
The Status: Successor Failure)
-
Status message: Descriptive error if the request failed.
If you do not see the job, look for this file:
<MFT-Install>logs/message/Scheduler-yyyy-mm-dd.txt
A summary record is written to this file for each job executed.
Next check if the job is currently running:
Go to Management > Scheduler > Jobs > Active Jobs.
This display lists the jobs currently running.
Next display the configured jobs:
Go to Management > Scheduler > Jobs > Manage Jobs.
Use the results table to limit the jobs that are returned.
The results table shows two important fields:
Field | Description |
---|---|
Last Fire Time | The last time the job was fired (i. e. executed) |
Next Fire Time | The next time the job will be fired (i. e. executed) |
A JMS initiated Transfer request does not execute
Command Center allows users to initiate Internet Server or Platform Server transfers by submitting XML to a JMS queue. Command Center might not execute a transfer due to the following reasons:
-
Verify that Command Center can connect to JMS and can listen on the queue.
-
Go to Management > Command Center Services > Configure JMS Service.
-
Select the Command Center instance where the test runs and click Test.
-
MFT attempts to connect to the target JMS server for all of the Command Center queue and topics. If this fails, a generic message is displayed. You need to look in the catalina.out to see the actual cause of the failure.
-
Check the following:
-
The JMS jar files are copied to:
<MFT-Install>/server/webapps/cfcc/WEB-INF/lib
-
-
After copying these files, MFT must be restarted.
-
The JMS Server URL is correct. Check the DNS Name and the port.
-
Some other process might be reading the JMS Queue. Use a JMS utility to verify the number of listeners for the queue.
-
Make sure that the request is written to the JMS Queue defined in: Configure JMS Service: Transfer and Job Request Properties: Queue Name
-
If the JMS request message stays in the queue and is not consumed, make sure the JMS request was initiated with the correct property. These properties are used as selectors by the Command Center JMS Service:
-
Platform Transfers: Platform-Request
-
Internet Transfers: Internet-Request
-
-
If necessary, enable tracing on the JMS Service
-
Go to Management > Command Center Services > JMS Service > Configure JMS Service.
-
Set Trace level to
All Messages
. -
Execute the process that adds the message to the JMS Queue. Two types of trace field are written:
-
EMS-Server…-trace-MFT-app-yyyy-mmm-dd.txt
-
-
This displays the date read from the JMS queue.
JMSServlet-trace-MFT-app-yyyy-mm-dd.txt
-
This file shows the processing performed for the requested function.
FTP Client directory list or transfer requests are failing
This assumes that the FTP Client can connect to the MFT FTP server and authenticate to the MFT FTP server. FTP transfers are susceptible to problems due to the requirement for two TCP connections for directory list, put, and get requests.
When a directory list fails, it generally means that the data connection has timed out or has been rejected. When a data connection fails, get and put file transfer requests also fail.
Here are some things that should be checked.
-
Check that firewall ports are defined correctly. In particular, when using a non-standard port (other than 21), the firewall needs to be configured for FTP processing.
-
The MFT FTP Service has two parameters that must be configured correctly.
-
Go to Administration > Transfer Servers > Configure FTP Server.
-
Set External IP Address to
Yes
. -
Set External IP Address to the IP address or IP name of the Internet Server host.
-
Go to Configuration > System Configuration.
-
In Global FTP settings, limit Local Ports Set to
Yes
. -
Starting Port Set to a number like 40000 or 30000. Along with the "Number of Ports to use", this defines the local TCP Ports that are used for data connections. You might need to supply this information to the firewall admins at the client and server side.
-
The number of ports to use are defined by how many TCP ports can be used for FTP data connections.
-
Generally speaking, PASV mode is the simpler to get working with firewalls. Hence, it is probably better for clients to use PASV mode when connecting to the MFT FTP server.
-
MFT supports two types of FTPS connections:
-
Explicit Clients connect to the Clear text FTP Port (usually 21) and issue a PORT command to enter SSL mode. The PORT command must be received before MFT processes the credentials.
-
Implicit Clients connect to the FTP Implicit SSL port. All traffic is SSL-encrypted.
-
-
When using Implicit SSL mode, make sure the client is connecting to the correct port. The default port for Implicit SSL is 990.