Jump to content


Photo

Essentials for email security mail delays (11/2/2016)


  • This topic is locked This topic is locked
4 replies to this topic

#1 Stephen Gee

Stephen Gee
  • Moderators
  • 83 posts

Posted 02 November 2016 - 10:37 AM

As of 12:00AM PDT, Barracuda is aware that some Essentials for Email Security customers are experiencing delays in email delivery. Customers may also experience delays in quarantine notifications and in viewing message logs. Both Barracuda Operations and engineering teams are actively working on resolving the issue at this time.

Please check status.barracuda.com for further updates.


  • ISA likes this

#2 Stephen Gee

Stephen Gee
  • Moderators
  • 83 posts

Posted 02 November 2016 - 02:09 PM

Some customers using the Barracuda Essentials for Email Security solution are experiencing delays in incoming email delivery, accessing the message log and end user quarantine interfaces. Other parts of the administrative user interface are fully accessible.  We are actively working on the problem. Our initial investigations revealed an unusually high volume of inbound connections from multiple unverified source IPs. We are in the process of sanitizing this traffic. As a result, the quality of service is gradually improving.

 

Our first priority is to restore the services to full capacity. We will provide more information as it becomes available.  Thank you for your patience.



#3 Stephen Gee

Stephen Gee
  • Moderators
  • 83 posts

Posted 02 November 2016 - 05:36 PM

Update as of 3:35PM PDT: Since 7PM PDT 11/1/2016, the Barracuda Essentials for Email Security and Cloud Protection Layer services have been experiencing a sustained and unusually large number of inbound connections from unverified sources. This resulted in some customers experiencing delays in email delivery and issues accessing the message/quarantine logs. We have made significant progress sanitizing these connections. At this point, email processing capabilities have returned to normal and backlogged messages are being delivered. Also, access to the message and quarantine logs is fully restored. We are continuously monitoring the connection load, taking further steps to mitigate the impact and tracing the root cause. Our priority is to restore the service to full capacity. We will provide more information through the course of the investigation. Thank you for your patience.



#4 Stephen Gee

Stephen Gee
  • Moderators
  • 83 posts

Posted 02 November 2016 - 07:34 PM

Update as of 11/2/2016 5:35PM PDT: Barracuda Networks is still continuing to see a large number of inbound connections from unverified sources for customers using Essentials for Email Security and Cloud Protection Layer.  We have successfully filtered and are actively monitoring the situation while taking the appropriate actions when needed.  Email processing has returned to normal.  Previously delayed  emails are now being accepted and processed.  We are continuing our investigation and will notify our customers once an analysis has been completed.  Thank you for your patience.



#5 Stephen Gee

Stephen Gee
  • Moderators
  • 83 posts

Posted 03 November 2016 - 06:13 PM

At 7PM PDT on Tuesday 11/1, the Barracuda Essentials for Email Security service and Cloud Protection Layer experienced elevated bandwidth utilization caused by an unusually high volume of unsolicited inbound DNS responses appearing to be from thousands of globally distributed hosts.  This traffic impacted email delivery, message log, and quarantine logs for some customers.
 
The Barracuda Operations team began deploying defensive measures immediately. These measures restored mail flow through the day as we mitigated the impact of the increased traffic load.
 
At this time, all systems are fully operational and the mail backlog had cleared as of yesterday afternoon.  We are closely monitoring the situation and implementing additional measures to strengthen our infrastructure.
 
While mail delivery was delayed for some customers, no email was lost, threat scanners were operational, and the administrative user interface was accessible throughout the troubleshooting process.  Emails were processed without any data loss and normal delivery resumed as we deployed additional mitigations.