Troubleshoot email delivery using the Exchange Online message trace tool

Office 365 Small Business admins can troubleshoot email delivery problems by using the Exchange Online message trace tool. The tool helps admins track specific messages sent in the past 90 days.

Sometimes an email message gets lost in transit, or it can take a lot longer than expected for delivery, and your users can wonder what happened. The message trace feature lets you follow messages as they pass through your Exchange Online service. With message tracing, you can determine whether a targeted email message was received, rejected, deferred, or delivered by the service. It also shows what events have occurred to the message before reaching its final status. Getting detailed information about a specific message lets you efficiently answer your user’s questions, troubleshoot mail flow issues, validate policy changes, and alleviates the need to contact technical support for assistance.

After you run a message trace, try to diagnose and resolve the problem yourself using the information that the message trace tool provides. If you are unable to resolve the problem, post a question to the Office 365 Community, or ask for customer support by filling a service request. For information about how to open up a service request for a message delivery issue, see Still can’t figure out what went wrong.

 Note    Office 365 users and admins can also use delivery reports in Outlook Web App to check the delivery status of messages they’ve sent or received. For example, if someone has sent a message to five people, they can check the status of the delivery of that message to each person. If people need more help, they can send the delivery report to their admin to troubleshoot using the message trace tool. Admins should also educate people in their organization about spam and virus protection in Office 365 Small Business. Having trouble getting email set up after adding your custom domain? See Troubleshoot email issues after you add your custom domain in Office 365.

In this article


Open the message trace tool

If you’re an Office 365 Small Business admin, do the following:

  1. Go to Admin > Service settings > Email, calendar, and contacts.
  2. Under Email troubleshooting, click Troubleshoot message delivery.

If you’re an Office 365 Midsize or Office 365 Enterprise admin, you access and run the message trace tool through the Exchange admin center.

Run a message trace

  1. In the message trace tool, depending on what you are searching for, you can enter values in the following fields. None of these fields is required. You can simply click Search to retrieve all message trace data for the default time period, which is the last 48 hours.
  • Date range   Using the drop-down list, select to search for messages sent or received within the past 24 hours, 48 hours, or 7 days. You can also select a custom time frame that includes any range within the past 90 days. For custom searches you can also change the time zone, in Coordinated Universal Time (UTC).
  • Delivery status   Using the drop-down list, select the status of the message you want to view information about. Leave the default value of All to cover all statuses. Other possible values are:
  • Delivered   The message or messages were successfully delivered to the intended destination.
  • Failed   The message was not delivered. Either it was attempted and failed, or it was not delivered as a result of actions taken by the filtering service. For example, if the message was determined to contain malware.
  • Pending   Delivery of the message is being attempted or re-attempted.
  • Expanded   The message was sent to a distribution group and was expanded so the members of the group can be viewed individually.
  • Unknown   The delivery status is unknown at this time. When the results of the query are listed, the delivery details fields will not content any information.

 Note    If you are running a message trace for items that are greater than 7 days old, you cannot select Pending or Unknown.

  • Message ID   This is the Internet message ID (also known as the Client ID) found in the header of the message with the “Message-ID:” token. Users can provide you with this information in order to investigate specific messages

The format of this ID varies depending on the sending mail system. The following is an example: <08f1e0f6806a47b4ac103961109ae6ef@server.domain>.

 Note    Be sure to include the full Message ID string. This may include angle brackets (<>).

This ID should be unique; however, it is dependent on the sending mail system for generation and not all sending mail systems behave the same way. As a result, there’s a possibility that you may get results for multiple messages when querying upon a single Message ID.

  • Sender   You can narrow the search for specific senders by clicking the Add sender button next to the Sender field. In the subsequent dialog box, select one or more senders from your company from the user picker list and then click Add. To add senders who aren’t on the list, type their email addresses and click Check names. In this box, wildcards are supported for email addresses in the format: *@contoso.com. When specifying a wildcard, other addresses can't be used. When you’re done with your selections, click OK.
  • Recipient   You can narrow the search for specific recipients by clicking the Add recipient button next to the Recipient field. In the subsequent dialog box, select one or more recipients from your company from the user picker list and then click Add. To add recipients who aren’t on the list, type their email addresses and click Check names. In this box, wildcards are supported for email addresses in the format: *@contoso.com. When specifying a wildcard, other addresses can't be used. When you’re done with your selections, click OK.
  1. If you’re searching for messages that are greater than 7 days old, specify the following parameter values (otherwise you can skip this step):
  • Include message events and routing details with report   We recommend selecting this check box only if you’re targeting one or a few specific messages, because including event details will result in a larger report that takes longer to process.
  • Direction   Using the drop-down list, select whether you want to search for All messages (this is the default), Inbound messages sent to your organization, or Outbound messages sent from your organization.
  • Original client IP address   Specify the IP address of the sender’s client.
  • Report title   Specify the unique identifier for this report. This will also be used as the subject line text for the email notification. The default is “Message trace report <day of the week>, <current date> <current time>”. For example, “Message trace report Thursday, October 17, 2013 7:21:09 AM”.
  • Notification email address   Specify the email address that you want to receive the notification when the message trace completes. This address must reside within your list of accepted domains.
  1. Click Search to run the message trace.

To search for a different message, you can click the Clear button and then specify new search criteria.

Top of Page Top of Page

View message trace results for messages that are less than 7 days old

After you’ve run the message trace, the results will be listed, sorted by date, with the most recent message appearing first. You can sort on any of the listed fields by clicking a column header. Clicking a column header again reverses the sort order. The following information is provided for each message:

  • Date   The date and time the message was received by the service, using the configured UTC time zone.
  • Sender   The email address of the sender in the format alias@domain.
  • Recipient   The email address of the recipient or recipients. For messages sent to more than one recipient, there is one line per recipient. If a recipient is a distribution group, the distribution group will be the first recipient, and then each member of the distribution group will be included on a separate line so you can check status for all recipients.
  • Subject   The subject line of the message. If necessary, this is truncated to the first 256 characters.
  • Status   This field specifies whether the message was Delivered to the recipient or the intended destination, Failed to be delivered to the recipient (either because it failed to reach its destination or because it was filtered), is Pending delivery (it is either in the process of being delivered or the delivery was deferred but is being re-attempted), was Expanded (there was no delivery because the message was sent to a distribution list (DL) that was expanded to the recipients of the DL), or has a status of None (there is no status of delivery for the message to the recipient because the message was either rejected or redirected to a different recipient).

 Note    The message trace can display a maximum of 500 entries. By default, the user interface displays 50 entries per page, and you can navigate through the pages. You can also change the entry size of each page up to 500.

Top of Page Top of Page

View details about a specific message that is less than 7 days old

After you review the list of items returned by running the message trace, you can double-click an individual message to view the following additional details about the message:

  • Message size   The size of the message, including attachments, in kilobytes (KB), or, if the message is bigger than 999 KB, in megabytes (MB).
  • Message ID   This is the Internet message ID (also known as the Client ID) found in the header of the message with the “Message-ID:” token. The form of this varies depending on the sending mail system. The following is an example: <08f1e0f6806a47b4ac103961109ae6ef@server.domain>.

This ID should be unique, however, it is dependent on the sending mail system for generation and not all sending mail systems behave the same way. As a result, there is a possibility that you may get results for multiple messages when querying upon a single Message ID.

This is given as output so that trace entries and the messages in question can be co-related.

  • To IP   The IP address or addresses to which the service attempted to deliver the message. If there is more than one recipient, all the IP addresses are displayed. For inbound messages sent to Exchange Online, this value is blank.
  • From IP   The IP address of the computer that sent the message. For outbound messages, this value is blank.
  • Delivery status   This field shows delivery status.

In the events section, the following fields provide information about the events that occurred to the message as it passed through the messaging pipeline:

  • Date   The date and time that an event occurred.
  • Event   This field briefly tells you what happened, for example, if the message was received by the service, if it was delivered or failed to be delivered to the intended recipient, and so on. The following are examples of events that may be listed:
    • RECEIVE   The message was received by the service.
    • SEND   The message was sent by the service.
    • FAIL   The message wasn’t delivered.
    • DELIVER   The message was delivered to a mailbox.
    • EXPAND   The message was sent to a distribution group that was expanded.
    • TRANSFER   Recipients were moved to a bifurcated message because of content conversion, message recipient limits, or agents.
    • DEFER   Delivery was postponed and may be re-attempted later.
  • Action   This field shows what action was performed if the message was filtered. For example, it may say a message was detected as spam and deleted.
  • Detail   This field provides more details about what happened. For example, it may tell you which specific malware was detected in which specific attachment, why a message was detected as spam, or that a message is extremely large. If the message was successfully delivered, it can tell you the IP address to which it was delivered.

Top of Page Top of Page

View the results of the message trace that is greater than 7 days old

If you run a message trace for items that are greater than 7 days old, when you click Search a message should appear letting you know that the message was successfully submitted, and that an email notification will be sent to the supplied email address when the trace has completed. (If the message trace is processed and data that matches your search criteria is successfully retrieved, this notification message will include information about the trace and a link to the downloadable .CSV file. If no data was found that matched the search criteria you specified, you’ll be asked to submit a new request with changed criteria in order to obtain valid results.)

In the message trace tool, you can click View pending or completed traces in order to view a list of traces that were run for items that are greater than 7 days old. In the resulting UI, the list of traces is sorted based on the date and time that they were submitted, with the most recent submissions appearing first. In addition to the report title, the date and time the trace was submitted, and the number of messages in the report, the following status values are listed:

  • Not started   The trace was submitted but is not yet running. At this point, you have the option to cancel the trace.
  • Cancelled   The trace was submitted but was cancelled.
  • In progress   The trace is running and you cannot cancel the trace or download the results.
  • Completed   The trace has completed and you can click Download this report to retrieve the results in a .CSV file. Note that if your message trace results exceed 5000 messages for a summary report, it will be truncated to the first 5000 messages. If your message trace results exceed 3000 messages for a detailed report, it will be truncated to the first 3000 messages. If you do not see all the results that you need, we recommend that break your search out into multiple queries.

When you select a specific message trace, additional information appears in the right pane. Depending on what search criteria you specified, this may include details such as the date range for which the trace was run, and the sender and intended recipients of the message.

 Note    Message traces containing data that is greater than 7 days old are automatically deleted. They cannot be manually deleted.

Top of Page Top of Page

View report details about a message that is greater than 7 days old

When you download and view a message trace report, either from the View pending or completed traces in the message trace tool or from a notification email, its contents depend on whether you have selected the Include message events and routing details with report option.

View a message trace report without routing details

If you didn’t include routing details when running the message trace, the following information is included in the .CSV file, which you can open in an application such as Microsoft Excel:

  • origin_timestamp   The date and time at which the message was received by the service, using the configured UTC time zone.
  • sender_address   The email address of the sender in the form alias@domain.
  • Recipient_status   The status of the delivery of the message to the recipient. If the message was sent to multiple recipients, it will show all the recipients and the corresponding status against each, in the format: <email address>##<status>. For example, a status of:

##Receive, Send means that the message was received by the service and sent to the intended destination.

##Receive, Fail means that the message was received by the service but failed to be delivered to the intended destination.

##Receive, Deliver means that the message was received by the service and delivered to the recipient’s mailbox.

  • message_subject   The subject line text of the message. If necessary, this is truncated to the first 256 characters.
  • total_bytes   The size of the message, including attachments, in bytes.
  • message_id   This is the Internet message ID (also known as the Client ID) found in the header of the message with the “Message-ID:” token. The form of this varies depending on the sending mail system. The following is an example: <08f1e0f6806a47b4ac103961109ae6ef@server. domain>.

This ID should be unique, however, it is dependent on the sending mail system for generation and not all sending mail systems behave the same way. As a result, there is a possibility that you may get results for multiple messages when querying upon a single Message ID.

This is given as output so that trace entries and the messages in question can be co-related.

  • network_message_id   This is a unique message ID value that persists across copies of the message that may be created due to bifurcation or distribution group expansion. An example value is 1341ac7b13fb42ab4d4408cf7f55890f.
  • original_client_ip   The IP address of the sender’s client.
  • Directionality   This field denotes whether the message was sent inbound (1) to your organization, or whether it was sent outbound (2) from your organization.
  • connector_id   The name of the source or destination Send connector or Receive connector. For example, ServerName\ConnectorName or ConnectorName.
  • delivery priority   Denotes whether the message was sent with High, Low, or Normal priority.

View a message trace report with routing details

If you included routing details when running the message trace, all information from the message tracking logs is included in the .CSV file, which you can open in an application such as Microsoft Excel. Some of the values included in this report are described in the prior section, while other values that may be useful for investigative purposes are described in the “Fields in the message tracking log files” section in the Message Tracking topic.

The custom_data field

Additionally, the custom_data field may contain values that are specific to the filtering service. The custom_data field in an AGENTINFO event is used by a variety of different agents to log details from the agent’s processing of the message. Some of the message data protection related agents are described below.

Spam Filter Agent (S:SFA)

A string beginning with S:SFA is an entry from the spam filter agent and provides the following key details:

Log Information Description
SFV=NSPM The message was marked as non-spam and was sent to the intended recipients.
SFV=SPM The message was marked as spam by the content filter.
SFV=BLK Filtering was skipped and the message was blocked because it originated from a blocked sender.
SFV=SKS The message was marked as spam prior to being processed by the content filter. This includes messages where the message matched a Transport rule to automatically mark it as spam and bypass all additional filtering.
SCL= <number> For more information about the different SCL values and what they mean, see Spam Confidence Levels.
PCL= <number> The Phishing Confidence Level (PCL) value of the message. These can be interpreted the same way as the SCL values documented in Spam Confidence Levels.
DI=SB The sender of the message was blocked.
DI=SQ The message was quarantined.
DI=SD The message was deleted.
DI=SJ The message was sent to the recipient’s Junk Email folder.
DI=SN The message was routed through the high risk delivery pool. For more information, see High Risk Delivery Pool for Outbound Messages.
DI=SO The message was routed through the normal outbound delivery pool.
SFS=[a]|SFS=[b] This denotes that spam rules were matched.
IPV=CAL The message was allowed through the spam filters because the IP address was specified in an IP Allow list in the connection filter.
H=[helostring] The HELO or EHLO string of the connecting mail server.
PTR=[ReverseDNS] The PTR record of the sending IP address, also known as the reverse DNS address.

When a message is filtered for spam, a sample custom_data entry would look similar to the following:

S:SFA=SUM|SFV=SPM|IPV=CAL|SRV=BULK|SFS=470454002|SFS=349001|SCL=9|SCORE=-1|LIST=0|DI=SN|RD=ftmail.inc.com|H=ftmail.inc.com|CIP=98.129.140.74|SFP=1501|ASF=1|CTRY=US|CLTCTRY=|LANG=en|LAT=287|LAT=260|LAT=18;

Malware Filter Agent (S:AMA)

A string beginning with S:AMA is an entry from the anti-malware agent and provides the following key details:

Log Information Description
AMA=SUM|v=1| or AMA=EV|v=1 The message was determined to contain malware. SUM denotes that the malware could’ve been detected by any number of engines. EV denotes that the malware was detected by a specific engine. When malware is detected by an engine this triggers the subsequent actions.
Action=r The message was replaced.
Action=p The message was bypassed.
Action=d The message was deferred.
Action=s The message was deleted.
Action=st The message was bypassed.
Action=sy The message was bypassed.
Action=ni The message was rejected.
Action=ne The message was rejected.
Action=b The message was blocked.
Name=<malware> The name of the malware that was detected.
File=<filename> The name of the file that contained the malware.

When a message contains malware, a sample custom_data entry would look similar to the following:

S:AMA=SUM|v=1|action=b|error=|atch=1;S:AMA=EV|engine=M|v=1|sig=1.155.974.0|name=DOS/Test_File|file=filename;S:AMA=EV|engine=A|v=1|sig=201307282038|name=Test_File|file=filename

Transport Rule Agent (S:TRA)

A string beginning with S:TRA is an entry from the transport rule agent and provides the following key details:

Log Information Description
ETR|ruleId=[guid] The rule ID that was matched.
St=[datetime] The date and time (in UTC) when the rule match occurred.
Action=[ActionDefinition] The action that was applied.
Mode=Enforce

The mode of the rule. Possible values are:

Enforce: All actions on the rule will be enforced.

Test with Policy Tips: Any Policy Tip actions will be sent, but other enforcement actions will not be acted on.

Test without Policy Tips: Actions will be listed in a log file, but senders will not be notified in any way, and enforcement actions will not be acted on.

When a message matches a transport rule, a sample custom_data entry would look similar to the following:

S:TRA=ETR|ruleId=19a25eb2-3e43-4896-ad9e-47b6c359779d|st=7/17/2013 12:31:25 AM|action=ApplyHtmlDisclaimer|sev=1|mode=Enforce

Top of Page Top of Page

Troubleshoot the problem using the results

So why didn’t someone receive a message they were expecting? Why did someone get a non-delivery report (NDR) for a message they sent?

Here’s where the troubleshooting comes in!

Use the various fields in the message trace tool to hone in on the cause of a delivery failure. For example, you probably know who sent the message to whom, and the general time the message was sent, so that’s a good place to start:

  1. Select a sender, a recipient or recipients, and the time interval, and run the message trace.
  2. In the list of results, look for messages with a delivery status of Failed or Pending. If you’re viewing a downloaded report, check the Recipient status field.
  3. Double-click the message you’re interested in to view the details:
  • The Events section explains why a message hasn’t been delivered.
  • The Detail column for a specific event may explain why the message wasn’t received. Check to see if the message was sent, if it was successfully received by Exchange Online, if it was filtered or redirected by the filtering service, or if it was subject to any delivery failures or delays. The Detail column also tells you if the message is extremely large or the destination isn’t responsive.
  • The Date column helps you follow the message through the messaging pipeline and indicates how long the service takes to process each event.

Top of Page Top of Page

Still can’t figure out what went wrong?

Post a question to the Office 365 Community, or ask for customer support by filling a service request, as follows:

  1. Go to Admin > Support > New service request.
  2. On the Identify issue page, enter the following information, and then click Next:
  • Issue type: Technical Support
  • Service: Exchange Online for Office 365 for Small Businesses
  • Service area: Mail Flow
  • Problem description: Message trace
  • Domain: As appropriate
  • Operating system: As appropriate
  • Microsoft Office version: As appropriate
  • Browser: As appropriate
  1. On the Add details page, describe your issue, using the information you’ve collected from the message trace, and then click Next.
  2. On the Attach file page, you can attach up to five screen shots or other documents. Then click Submit.
  3. You’ll get confirmation of the request and a link on the Service requests page to follow the status of the request.

Top of Page Top of Page

Message trace FAQ

ShowAfter a message is sent, how long before a message trace can pick it up?

Message trace data can appear as soon as 10 minutes after a message is sent, or it can take up to one hour.

ShowWhy am I getting a timeout error when I run a message trace?

The search is probably taking too long. Try simplifying your search criteria.

ShowWhy is my message taking so long to arrive to its destination?

Possible causes include the following:

  • The intended destination isn’t responsive. This is the most likely scenario.
  • A large message takes a long time to process.
  • Latency in the service is causing delays.
  • The message was blocked by the filtering service.

Follow the instructions in the View details about a specific message section of this article to understand the possible causes.

Top of Page Top of Page

 
 
Applies to:
Office 365 operated by 21Vianet - Small Business admin, Office 365 Small Business admin