tag:status.secureframe.com,2005:/historySecureframe Status - Incident History2024-03-28T08:47:43-07:00Secureframetag:status.secureframe.com,2005:Incident/200890732024-02-27T10:25:30-08:002024-02-27T10:25:30-08:00Degraded Service due to backed up jobs<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>10:25</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>09:35</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>08:05</var> PST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being applied</p>tag:status.secureframe.com,2005:Incident/200224922024-02-19T10:56:16-08:002024-02-19T10:56:16-08:00Security Questionnaires Not Responding<p><small>Feb <var data-var='date'>19</var>, <var data-var='time'>10:56</var> PST</small><br><strong>Resolved</strong> - A fix was shipped to our production system to resolve this issue.</p><p><small>Feb <var data-var='date'>19</var>, <var data-var='time'>09:36</var> PST</small><br><strong>Investigating</strong> - We are currently investigating an issue that is preventing Security Questionnaires from loading properly on the Secureframe site.</p>tag:status.secureframe.com,2005:Incident/199223892024-02-06T07:19:17-08:002024-02-06T07:19:17-08:00Sign in to Employee Onboarding for a subset of users is not working<p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>07:19</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>07:09</var> PST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.secureframe.com,2005:Incident/198593932024-01-29T11:47:57-08:002024-01-29T11:47:57-08:00Employee onboarding redirect issues<p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>11:47</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>11:43</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>08:35</var> PST</small><br><strong>Identified</strong> - We're experiencing issues loading employee onboarding for customers that have policies enabled and do not have trainings enabled.</p>tag:status.secureframe.com,2005:Incident/189812182023-10-31T13:47:28-07:002023-10-31T13:47:28-07:00Intermittent Dashboard Errors<p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>13:47</var> PDT</small><br><strong>Resolved</strong> - We have completed monitoring.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>10:40</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>10:34</var> PDT</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>10:30</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.secureframe.com,2005:Incident/189686432023-10-30T08:29:31-07:002023-10-30T08:29:31-07:00Intermittent slowness<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>08:29</var> PDT</small><br><strong>Resolved</strong> - We are back to normal operation.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>08:22</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we're monitoring the application</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>07:47</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and we are working on a fix</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>07:40</var> PDT</small><br><strong>Investigating</strong> - We're experiencing an intermittent errors and slower loading pages. We are currently looking into the issue.</p>tag:status.secureframe.com,2005:Incident/186804532023-10-03T12:50:18-07:002023-10-03T12:50:18-07:00General Page Performance Issues<p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>12:50</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>12:16</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>11:31</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>11:19</var> PDT</small><br><strong>Investigating</strong> - We have identified a performance issue affecting various pages on the site, and are deploying a fix now.</p>tag:status.secureframe.com,2005:Incident/181079582023-08-10T11:09:04-07:002023-08-10T11:09:04-07:00Personnel Pages - Empty Display of Personnel<p><small>Aug <var data-var='date'>10</var>, <var data-var='time'>11:09</var> PDT</small><br><strong>Resolved</strong> - Production fixed has been rolled out, and normal operations have resumed.</p><p><small>Aug <var data-var='date'>10</var>, <var data-var='time'>10:42</var> PDT</small><br><strong>Identified</strong> - Some Secureframe users may see an empty Personnel page in the application. We have determined the cause, and we are deploying an expedited bug-fix for this issue.</p>tag:status.secureframe.com,2005:Incident/178892502023-07-18T07:00:49-07:002023-07-18T07:00:49-07:00Azure Cloud integrations not connecting<p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>07:00</var> PDT</small><br><strong>Resolved</strong> - A fix has been implemented and we are now looking at forcing an Azure re-sync for customers.</p><p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>06:38</var> PDT</small><br><strong>Investigating</strong> - Azure Cloud integrations are currently not connecting. We are investigating the issue.</p>tag:status.secureframe.com,2005:Incident/177910952023-07-07T09:04:42-07:002023-07-07T09:04:42-07:00Integration Syncs Delayed<p><small>Jul <var data-var='date'> 7</var>, <var data-var='time'>09:04</var> PDT</small><br><strong>Resolved</strong> - All remaining daily syncs are running normally and will complete within the next 1-2 hours.</p><p><small>Jul <var data-var='date'> 7</var>, <var data-var='time'>08:48</var> PDT</small><br><strong>Monitoring</strong> - We have alleviated the main cause of delays. Some nightly integration sync jobs are still completing.</p><p><small>Jul <var data-var='date'> 7</var>, <var data-var='time'>08:16</var> PDT</small><br><strong>Update</strong> - We have increased system resources to improve sync times: this should allow syncing to complete more quickly and reduce overall latency.</p><p><small>Jul <var data-var='date'> 7</var>, <var data-var='time'>08:01</var> PDT</small><br><strong>Investigating</strong> - We are investigating this issue.</p>tag:status.secureframe.com,2005:Incident/177752112023-07-05T16:11:26-07:002023-07-05T16:11:26-07:00Errors with Trust Center Pages<p><small>Jul <var data-var='date'> 5</var>, <var data-var='time'>16:11</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'> 5</var>, <var data-var='time'>16:09</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.secureframe.com,2005:Incident/174823302023-06-05T15:58:06-07:002023-06-05T16:19:35-07:00Increased Errors on WAFV2: GetLoggingConfiguration Findings for AWS Connections<p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>15:58</var> PDT</small><br><strong>Resolved</strong> - We have resolved the underlying issue and all systems are operating normally. There may be some lingering errors in test results that will be resolved as our AWS syncing runs.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>14:06</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate this issue. We have taken remediation steps to reduce and/or resolve test results that incorrectly show failed results, and as AWS accounts sync, these cosmetic issues should resolve themselves.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>13:28</var> PDT</small><br><strong>Update</strong> - This error can show across all AWS tests in some cases. We are continuing to investigate a solution.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>13:13</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate this error.</p><p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>12:55</var> PDT</small><br><strong>Investigating</strong> - We are investigating an increased number of incorrect findings in our AWS test findings, specifically WAFV2: GetLoggingConfiguration.</p>tag:status.secureframe.com,2005:Incident/167406122023-04-03T13:33:13-07:002023-04-03T13:33:13-07:00Degraded Performance: Manual Integration Sync, Test Refreshing<p><small>Apr <var data-var='date'> 3</var>, <var data-var='time'>13:33</var> PDT</small><br><strong>Resolved</strong> - We temporarily increased infrastructure resources in order to reduce a backlog of background jobs that were preventing integration syncing and test refreshes. This incident is resolved, and we are continuing to investigate root causes and monitoring.</p><p><small>Apr <var data-var='date'> 3</var>, <var data-var='time'>12:45</var> PDT</small><br><strong>Investigating</strong> - Some customers may experience delays when attempting to manually sync integrations, or refresh tests in the Secureframe app. We are investigating the incident.</p>tag:status.secureframe.com,2005:Incident/161551262023-02-15T16:21:57-08:002023-02-15T16:21:57-08:00Secureframe Not Reachable<p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>16:21</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>15:53</var> PST</small><br><strong>Investigating</strong> - We have received reports of customers not being able to load part or all of Secureframe. We are investigating.</p>tag:status.secureframe.com,2005:Incident/158637202023-01-17T11:03:31-08:002023-01-17T11:03:31-08:00Secureframe - issues with updating Cloud Sync results<p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>11:03</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>11:02</var> PST</small><br><strong>Update</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>10:45</var> PST</small><br><strong>Update</strong> - GCP syncing should now be working normally,</p><p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>10:09</var> PST</small><br><strong>Update</strong> - We are continuing to work on this issue; AWS syncing should now function correctly, other cloud platforms are nearing resolution.</p><p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>09:06</var> PST</small><br><strong>Update</strong> - We are continuing to work on this issue - we are testing resolutions at this time.</p><p><small>Jan <var data-var='date'>17</var>, <var data-var='time'>08:03</var> PST</small><br><strong>Identified</strong> - Some customers may experience 'Sync Failed' messages in the application as well as test refreshing due to an outage in our backend storage system. We have identified a cause, and have taken steps to resolve this issue and are still actively investigating.</p>tag:status.secureframe.com,2005:Incident/152820482022-12-16T10:50:47-08:002022-12-16T10:50:47-08:00Secureframe Agent not installating on Windows and Linux<p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>10:50</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>10:36</var> PST</small><br><strong>Update</strong> - We have deployed a fix for installing agents. Existing agent packages did not have any issues with syncing or installation.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>14:39</var> PST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented. This only affects the generation of new agent packages.</p>tag:status.secureframe.com,2005:Incident/140368442022-11-30T16:05:10-08:002022-11-30T16:05:10-08:00Secureframe Not Reachable<p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>16:05</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>15:29</var> PST</small><br><strong>Monitoring</strong> - This issue has been resolved by the upstream provider, we will continue to monitor the application's performance. Application performance should be normal for all users at this time.</p><p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>15:15</var> PST</small><br><strong>Identified</strong> - We have received reports of customers not being able to load part or all of Secureframe. We are investigating. This is due to an upstream provider outage. We are monitoring those systems.</p>tag:status.secureframe.com,2005:Incident/117681082022-10-09T20:50:55-07:002022-10-09T20:50:55-07:00Planned System Maintenance and Upgrades<p><small>Oct <var data-var='date'> 9</var>, <var data-var='time'>20:50</var> PDT</small><br><strong>Resolved</strong> - We have completed maintenance</p><p><small>Oct <var data-var='date'> 9</var>, <var data-var='time'>17:48</var> PDT</small><br><strong>Update</strong> - Maintenance is progressing successfully. We're re-enabling most functionality on the application. <br /><br />The audit log page will continue to have degraded performance for the remainder of the maintenance.</p><p><small>Oct <var data-var='date'> 9</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>Monitoring</strong> - We're in the process of upgrading our services from 5:00 PM PDT - 8:00 PM PDT.</p>tag:status.secureframe.com,2005:Incident/112395832022-09-23T15:36:13-07:002022-09-23T15:36:13-07:00Employee onboarding is inaccessible for a subset of users<p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>15:36</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>15:36</var> PDT</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>15:35</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>14:57</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>14:31</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>14:00</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue</p>tag:status.secureframe.com,2005:Incident/110922872022-09-09T14:27:10-07:002022-09-09T14:27:10-07:00Elevated API Errors<p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>14:27</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>14:16</var> PDT</small><br><strong>Investigating</strong> - We're experiencing an elevated level of API errors and are currently looking into the issue.</p>tag:status.secureframe.com,2005:Incident/110291712022-08-31T17:04:17-07:002022-08-31T17:04:18-07:00Asset tables are not loading<p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>17:04</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>16:35</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.secureframe.com,2005:Incident/110248862022-08-31T05:24:28-07:002022-08-31T05:24:28-07:00Login to Secureframe failing for some customers<p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>05:24</var> PDT</small><br><strong>Resolved</strong> - We have resolved the issue</p><p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>05:16</var> PDT</small><br><strong>Investigating</strong> - We have received reports of customers not being able to log in to Secureframe. We are investigating.</p>tag:status.secureframe.com,2005:Incident/108953002022-08-23T17:26:32-07:002022-08-23T17:26:32-07:00DNS Issue with Heroku<p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>17:26</var> PDT</small><br><strong>Resolved</strong> - We have found a workaround to the hosting providers ongoing issue.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>17:04</var> PDT</small><br><strong>Identified</strong> - We are continuing the monitor the situation at our hosting provider. It's a widespread issue impacting their customers in the US and Europe. <br /><br />They are currently working on a solution https://status.heroku.com/</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>16:25</var> PDT</small><br><strong>Investigating</strong> - Our hosting provider is having a DNS outage. More information can be found here https://status.heroku.com/incidents/2453</p>tag:status.secureframe.com,2005:Incident/108082662022-08-11T15:12:57-07:002022-08-11T15:12:57-07:00Secureframe not loading for some customers<p><small>Aug <var data-var='date'>11</var>, <var data-var='time'>15:12</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>11</var>, <var data-var='time'>15:06</var> PDT</small><br><strong>Investigating</strong> - We have received reports of customers not being able to load part or all of Secureframe. We are investigating</p>tag:status.secureframe.com,2005:Incident/108066332022-08-11T10:45:28-07:002022-08-11T10:46:20-07:00Secureframe not loading for some customers<p><small>Aug <var data-var='date'>11</var>, <var data-var='time'>10:45</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>11</var>, <var data-var='time'>10:44</var> PDT</small><br><strong>Monitoring</strong> - Secureframe is now loading for all customers. We are still investigating why some images aren't loading for everyone.</p><p><small>Aug <var data-var='date'>11</var>, <var data-var='time'>10:40</var> PDT</small><br><strong>Investigating</strong> - We have received reports of customers not being able to load part or all of Secureframe. We are investigating.</p>