Incidents reported on status page for CilixCloud https://status.cilix.cloud/ https://d1lppblt9t2x15.cloudfront.net/logos/3a70572ce4abf82e367d9ca1145b1cd8.png https://status.cilix.cloud/ en https://status.cilix.cloud/ Fri, 22 May 2026 09:23:39 +0000 https://status.cilix.cloud/#5c7794f233253d87c403a23c842eff0b1fabb09805bb3d60a773d5446ac311a1 VPS - lon1-a5 recovered https://status.cilix.cloud/ Fri, 22 May 2026 08:59:06 +0000 https://status.cilix.cloud/#5c7794f233253d87c403a23c842eff0b1fabb09805bb3d60a773d5446ac311a1 VPS - lon1-a5 went down https://status.cilix.cloud/incident/902608 Fri, 22 May 2026 08:00:57 -0000 https://status.cilix.cloud/incident/902608#3228df63383cff584eaaadab88a589ac3da7d490f3335e53ea9cf76e82f1bd46 We will be migrating VPS services to THN2 from 0900 to 1300 on 22nd May 2026. We expect services to be restored in under 2 hours, however, we have allowed for 4 hours. https://status.cilix.cloud/ Thu, 21 May 2026 14:50:02 +0000 https://status.cilix.cloud/#568fab0e4e5af0c4d142bd88ed6a047f9905c3a3d5c7331b4ebf018d58eff4b5 DDoS Protection recovered https://status.cilix.cloud/ Thu, 21 May 2026 14:05:16 +0000 https://status.cilix.cloud/#568fab0e4e5af0c4d142bd88ed6a047f9905c3a3d5c7331b4ebf018d58eff4b5 DDoS Protection went down https://status.cilix.cloud/ Thu, 21 May 2026 07:45:06 +0000 https://status.cilix.cloud/#1e9e268ef95d3d0f441efd83bb9d16c246c413d968ab79d37978d1d99c445978 SMTP Relay 1 (London) recovered https://status.cilix.cloud/ Wed, 20 May 2026 17:55:53 +0000 https://status.cilix.cloud/#1e9e268ef95d3d0f441efd83bb9d16c246c413d968ab79d37978d1d99c445978 SMTP Relay 1 (London) went down https://status.cilix.cloud/ Wed, 20 May 2026 17:47:20 +0000 https://status.cilix.cloud/#87941df9d7bad7db569788150a6f804e2c4fad4a25dfccca50a9cdd892a6141e SMTP Relay 1 (London) recovered https://status.cilix.cloud/ Wed, 20 May 2026 17:19:53 +0000 https://status.cilix.cloud/#87941df9d7bad7db569788150a6f804e2c4fad4a25dfccca50a9cdd892a6141e SMTP Relay 1 (London) went down https://status.cilix.cloud/ Wed, 20 May 2026 17:05:39 +0000 https://status.cilix.cloud/#08e0dae3843d329f56ca568f69335eaf7379135a69d814f03c6add315a5116d9 Customer Portal recovered https://status.cilix.cloud/ Wed, 20 May 2026 16:46:01 +0000 https://status.cilix.cloud/#08e0dae3843d329f56ca568f69335eaf7379135a69d814f03c6add315a5116d9 Customer Portal went down https://status.cilix.cloud/ Wed, 20 May 2026 10:25:00 +0000 https://status.cilix.cloud/#d92e8aab90e21849407be3b92c9c92b3a8354ba0ba4b7542339966da9579adf5 Leased Lines (Ethernet Circuits) recovered https://status.cilix.cloud/ Wed, 20 May 2026 10:12:40 +0000 https://status.cilix.cloud/#d92e8aab90e21849407be3b92c9c92b3a8354ba0ba4b7542339966da9579adf5 Leased Lines (Ethernet Circuits) went down https://status.cilix.cloud/ Mon, 18 May 2026 13:05:02 +0000 https://status.cilix.cloud/#6088d254371cb2bfc0777c980828e0115f83358c0c63858f1aa2d51438ca6d71 DDoS Protection recovered https://status.cilix.cloud/ Sun, 17 May 2026 06:55:45 +0000 https://status.cilix.cloud/#6088d254371cb2bfc0777c980828e0115f83358c0c63858f1aa2d51438ca6d71 DDoS Protection went down https://status.cilix.cloud/ Sun, 10 May 2026 00:02:14 +0000 https://status.cilix.cloud/#0d7f917f6db539218687da4f0b5a1b43aaada2750b430bfea97dc5c443aa49b3 Leased Lines (Ethernet Circuits) recovered https://status.cilix.cloud/ Sat, 09 May 2026 23:56:05 +0000 https://status.cilix.cloud/#0d7f917f6db539218687da4f0b5a1b43aaada2750b430bfea97dc5c443aa49b3 Leased Lines (Ethernet Circuits) went down https://status.cilix.cloud/ Fri, 24 Apr 2026 18:15:02 +0000 https://status.cilix.cloud/#ca3a11d2c0846db092d6870a4d6cb6b5219e6519f194f924357aaa9bb3b1d25f DDoS Protection recovered https://status.cilix.cloud/ Fri, 24 Apr 2026 14:55:36 +0000 https://status.cilix.cloud/#ca3a11d2c0846db092d6870a4d6cb6b5219e6519f194f924357aaa9bb3b1d25f DDoS Protection went down https://status.cilix.cloud/ Sun, 12 Apr 2026 10:35:14 +0000 https://status.cilix.cloud/#e13b74a9ba8d1be77e800e3ed3f2cfbaf03f871c76e3e06c76c1f7aee801d2ac DDoS Protection recovered https://status.cilix.cloud/ Sun, 12 Apr 2026 08:00:36 +0000 https://status.cilix.cloud/#e13b74a9ba8d1be77e800e3ed3f2cfbaf03f871c76e3e06c76c1f7aee801d2ac DDoS Protection went down https://status.cilix.cloud/ Fri, 10 Apr 2026 11:44:19 +0000 https://status.cilix.cloud/#fa6b1192c95dbe563a933831d4998de20d55a74603efe91983dda66ca09bcc90 Leased Lines (Ethernet Circuits) recovered https://status.cilix.cloud/ Fri, 10 Apr 2026 10:51:08 +0000 https://status.cilix.cloud/#fa6b1192c95dbe563a933831d4998de20d55a74603efe91983dda66ca09bcc90 Leased Lines (Ethernet Circuits) went down https://status.cilix.cloud/ Fri, 10 Apr 2026 02:20:02 +0000 https://status.cilix.cloud/#5e61223feadaf322e2935963017afa2e9a9503b14e127e44d4403ad935aaeeda DDoS Protection recovered https://status.cilix.cloud/ Fri, 10 Apr 2026 00:50:39 +0000 https://status.cilix.cloud/#5e61223feadaf322e2935963017afa2e9a9503b14e127e44d4403ad935aaeeda DDoS Protection went down https://status.cilix.cloud/ Fri, 10 Apr 2026 00:20:02 +0000 https://status.cilix.cloud/#fa7c4aa57abbf2b16c3aadd4da7d35ac4473252eff153652bd9a69edc0237de9 DDoS Protection recovered https://status.cilix.cloud/ Thu, 09 Apr 2026 23:35:07 +0000 https://status.cilix.cloud/#fa7c4aa57abbf2b16c3aadd4da7d35ac4473252eff153652bd9a69edc0237de9 DDoS Protection went down https://status.cilix.cloud/ Thu, 09 Apr 2026 23:24:35 +0000 https://status.cilix.cloud/#8ddc6274ad5d868e21c5085c72f37a91f0a70405655f0875c09fc2e88e1532ec DDoS Protection recovered https://status.cilix.cloud/ Thu, 09 Apr 2026 22:30:43 +0000 https://status.cilix.cloud/#8ddc6274ad5d868e21c5085c72f37a91f0a70405655f0875c09fc2e88e1532ec DDoS Protection went down https://status.cilix.cloud/ Thu, 09 Apr 2026 16:35:02 +0000 https://status.cilix.cloud/#05b29197cfcea42c071a3ab507e65b8777f1340f29b89dbcfc21ba940063b396 DDoS Protection recovered https://status.cilix.cloud/ Thu, 09 Apr 2026 16:20:47 +0000 https://status.cilix.cloud/#05b29197cfcea42c071a3ab507e65b8777f1340f29b89dbcfc21ba940063b396 DDoS Protection went down https://status.cilix.cloud/ Wed, 08 Apr 2026 19:20:01 +0000 https://status.cilix.cloud/#ba84b167738c48c7e673b4ce574b981d5a87f7a4f45d79b67ffec1ba88145036 DDoS Protection recovered https://status.cilix.cloud/ Wed, 08 Apr 2026 18:55:43 +0000 https://status.cilix.cloud/#ba84b167738c48c7e673b4ce574b981d5a87f7a4f45d79b67ffec1ba88145036 DDoS Protection went down https://status.cilix.cloud/ Sat, 28 Mar 2026 04:53:01 +0000 https://status.cilix.cloud/#65ee8079d90e35f0685108447b90aa54126957279b071681e952f4fd1da0dd7f SMTP Relay 1 (London) recovered https://status.cilix.cloud/ Sat, 28 Mar 2026 04:45:56 +0000 https://status.cilix.cloud/#65ee8079d90e35f0685108447b90aa54126957279b071681e952f4fd1da0dd7f SMTP Relay 1 (London) went down https://status.cilix.cloud/ Sat, 28 Mar 2026 03:58:20 +0000 https://status.cilix.cloud/#6af2bf84e9fc44b82836a5a081c1534443191e9b2198a16b5be212079c184f63 Leased Lines (Ethernet Circuits) recovered https://status.cilix.cloud/ Sat, 28 Mar 2026 03:48:11 +0000 https://status.cilix.cloud/#6af2bf84e9fc44b82836a5a081c1534443191e9b2198a16b5be212079c184f63 Leased Lines (Ethernet Circuits) went down https://status.cilix.cloud/incident/845999 Tue, 10 Mar 2026 11:25:00 -0000 https://status.cilix.cloud/incident/845999#ac9b4c727f9d5bdf05740690d906527b78d9ba93904a85e284b9f4575a14cd97 This has been traced to a NIC issue in the impacted hypervisor host. The on-site team has replaced the impacted NIC, and affected services are now back online. https://status.cilix.cloud/incident/845999 Tue, 10 Mar 2026 11:00:00 -0000 https://status.cilix.cloud/incident/845999#5daf661be2d746f91e4a098235c83e444ce1fca8b9ee48e4bc439ce2bc9bc824 We are aware of an issue impacting the availability of Virtual Private Servers on node lon1-a5-u18. We are currently investigating. https://status.cilix.cloud/ Fri, 27 Feb 2026 01:48:31 +0000 https://status.cilix.cloud/#221361518663bc2dfb119bbb8d1dd62defd5c6baba906b3934f0a8248383dc5e Leased Lines (Ethernet Circuits) recovered https://status.cilix.cloud/ Fri, 27 Feb 2026 01:38:52 +0000 https://status.cilix.cloud/#221361518663bc2dfb119bbb8d1dd62defd5c6baba906b3934f0a8248383dc5e Leased Lines (Ethernet Circuits) went down https://status.cilix.cloud/incident/806114 Sat, 17 Jan 2026 02:00:48 -0000 https://status.cilix.cloud/incident/806114#28f8c7ad50c64c42a81ca55e8381e389d9f248a21dc13f56d0a6b2e7720a71ff Date: Saturday 17th January 2026 Time: 02:00 – 07:00 GMT Impact: Service interruption of up to 10 minutes during the maintenance window Reason: Configuration changes on border routers. This work will deliver improved capacity, enhanced resiliency, and greater technical flexibility for customer services. No action is required. Services will continue to operate as normal following completion of the work. We apologise for any inconvenience and thank you for understanding. https://status.cilix.cloud/ Sun, 21 Dec 2025 01:27:33 +0000 https://status.cilix.cloud/#9de23002c918bdf0d309d95ba4b3a38ba638063dd48b0df040523d2cadef5c96 Broadband Services recovered https://status.cilix.cloud/ Sun, 21 Dec 2025 00:18:38 +0000 https://status.cilix.cloud/#9de23002c918bdf0d309d95ba4b3a38ba638063dd48b0df040523d2cadef5c96 Broadband Services went down https://status.cilix.cloud/incident/756918 Sat, 20 Dec 2025 22:00:00 -0000 https://status.cilix.cloud/incident/756918#85e28ee5ecad3687a68ddf1f57e7d29337dc5c68bb16e44eab4d16df26bfba5a We will be migrating the compute and storage nodes in our London 2 location to a new rack in the data centre, dedicated to compute and storage. Customers will experience up to 30 minutes of disruption as compute and storage nodes are physically moved to the new rack, patched in and powered back on. Work is scheduled to begin at 0000 and will be completed by 0500. Updates will be provided on this page as required. https://status.cilix.cloud/incident/757840 Wed, 05 Nov 2025 19:08:00 -0000 https://status.cilix.cloud/incident/757840#4783329e4555f302a83de22cf7826302e9f9c389811b6da902893a1a89b57b0d Broadband services have been stable over the last 24 hours. https://status.cilix.cloud/incident/757840 Tue, 04 Nov 2025 18:42:00 -0000 https://status.cilix.cloud/incident/757840#d06f3971ed4b7273f2a8e33ec04b240b468d1b87ccc3bf0336e35372f1766378 We have seen a recurrence of the session & connectivity drop. We have informed ICUK (the wholesale provider), who are currently working towards service restoration. https://status.cilix.cloud/incident/757840 Tue, 04 Nov 2025 15:28:00 -0000 https://status.cilix.cloud/incident/757840#34e82d59dd66a26c4bdcddf213521d629ee1ddae51a298c94b50f91472afc594 All affected services are currently back online. We are working with ICUK closely to mitigate any further impact. Further updates will be provided as required. https://status.cilix.cloud/incident/757840 Tue, 04 Nov 2025 15:00:00 -0000 https://status.cilix.cloud/incident/757840#202d652d0b9fac42699cdd004ef356d3b8d33adeb845e687b51034363c1739e3 We are experiencing intermittent service as ICUK appears to be handling several denial-of-service attacks. Further information will be provided as required. https://status.cilix.cloud/incident/757840 Tue, 04 Nov 2025 11:40:00 -0000 https://status.cilix.cloud/incident/757840#2cf64e5d50aafd262a8d513929d6dfd2fb811de24ec93f62c67d55da73b9073e We are aware of a fault affecting broadband services via ICUK starting at 11:40 AM. We will update this page as more information is available. https://status.cilix.cloud/incident/751268 Sun, 26 Oct 2025 03:00:00 -0000 https://status.cilix.cloud/incident/751268#eee6bb129c4859482a4693e7dbd92021fe29b18e5616ce2ad81fdfa2ace36db0 We will be performing upgrades to our network infrastructure in our London 2 POP. Traffic will be diverted away as maintenance begins. https://status.cilix.cloud/incident/744286 Wed, 15 Oct 2025 20:37:00 -0000 https://status.cilix.cloud/incident/744286#269563829c7417181642d7291b4a7bf298ed7b18672cf7df3b18f7000805c6fd #RFO# ##Date and Time of Incident:## 15-Oct-2025, 20:28 - 20:49 ##Timeline of Events## - 20:28: Availability alerts triggered, indicating a loss of Web Application Firewall cluster 1. - 20:32: Technical teams begin fault investigation. - 20:40: Issue identified as a locked-up worker node. - 20:45: Worker node removed from WAF cluster. - 20:49: Service availability returns to normal. ##Summary of impact## One of the WAF worker nodes in cluster 1 experienced a software failure that caused the TCP session state table to fail synchronising correctly across the WAF cluster. This, in turn, led to the cluster falling out of synchronisation and failing to pass traffic correctly. Services routing through the affected WAF cluster were unavailable for 21 minutes. ##Resolution Status## We have multiple redundancies in place to prevent a total outage from happening should a worker node fail entirely; however, in this case, a worker node experienced a very unique partial failure, where it was sending OKs to health checks, but wasn’t healthy. We have engaged with our WAF vendors TAC, and will deploy remedial fixes as soon as they are made available. As a precaution, we will be reloading all worker nodes overnight. No downtime is expected from this change. https://status.cilix.cloud/incident/744286 Wed, 15 Oct 2025 19:49:00 -0000 https://status.cilix.cloud/incident/744286#1af03ef9ab6be097b4d5197fa364c644dd4c06aedf4776077de200fead070f3e Services are now back online. https://status.cilix.cloud/incident/744286 Wed, 15 Oct 2025 19:42:00 -0000 https://status.cilix.cloud/incident/744286#d136ce697763457c233ab395125f6ef88318ecdc5dfe0f84d65668a20791e4a5 We have identified the issue and are working to bring the services back online. https://status.cilix.cloud/incident/744286 Wed, 15 Oct 2025 19:32:00 -0000 https://status.cilix.cloud/incident/744286#a4ae49ddd1b57e2b6c3a8cf840c53f7aea065ba4477727f650ddccc4b4700d21 We are investigating availability issues with our Web Application Firewall. https://status.cilix.cloud/incident/740414 Fri, 10 Oct 2025 02:00:04 -0000 https://status.cilix.cloud/incident/740414#0e9be4633a0b99cc4573fb595a5008edbdc856eb136e60ef5973c04d6b8e95b9 ##What's Happening?## In light of the recent DDoS attacks against several large networks, causing major disruption, we will be making configuration, topology and capacity changes to our routing infrastructure in our London 2 location. These changes will significantly enhance our on-net DDoS mitigation capacity and will minimise the effects of any congestion in the event of a large DDoS attack. These changes are scheduled for 10-Oct-25 from 3 - 5 AM BST. ##What's the impact?## We will divert traffic away from our London 2 border routers before maintenance begins, meaning there should be little to no impact for most workloads. Any impact experienced should be brief (less than 1 minute) while traffic is rerouted over available paths. ##Updates## Updates will be provided on this maintenance note as required. https://status.cilix.cloud/incident/738761 Thu, 09 Oct 2025 07:45:00 -0000 https://status.cilix.cloud/incident/738761#d462e097a6b02c6ef1d93324a7767889df67bc1e2125cd2e211b74a19e2e8b17 The provider has confirmed this disruption was a result of a targeted DDoS attack against their core network. They have taken measures to filter the malicious traffic out and have seen full service restoration. https://status.cilix.cloud/incident/738761 Mon, 06 Oct 2025 21:23:00 -0000 https://status.cilix.cloud/incident/738761#a6493a5918c5a4a1c5ef691c5831f33ca7f7f4a00958b0edb527399974132aaf Services have been stable for the past 40 minutes. We will follow up with an RFO shortly. https://status.cilix.cloud/incident/738761 Mon, 06 Oct 2025 20:40:00 -0000 https://status.cilix.cloud/incident/738761#569c1262f7883866a78ffc76bc2314f7ce71d3048a86d0cd6faf33dc8b5f7cd5 We are seeing the majority of broadband sessions back online. https://status.cilix.cloud/incident/738761 Mon, 06 Oct 2025 19:25:00 -0000 https://status.cilix.cloud/incident/738761#9818d4c6014c8f59de4192e6ae5d9c2fc9c01d3dd8e0538c82ec39f6da077c3f The provider in question has acknowledged the issue and is working towards a resolution. https://status.cilix.cloud/incident/738761 Mon, 06 Oct 2025 19:00:00 -0000 https://status.cilix.cloud/incident/738761#14625414c5648ea3a52265d73b1925bed9156ac78d6a7614b3b555a6efd988bc We have identified the source of this fault to an upstream provider. We have engaged with the L2TP provider for further information. https://status.cilix.cloud/incident/738761 Mon, 06 Oct 2025 18:42:00 -0000 https://status.cilix.cloud/incident/738761#0c4f4dfcdee82e54de6fca664d280241456a918633c3ef3510b8123d447a6406 We are aware of a fault affecting L2TP dynamic broadband services starting at 19:42. L2TP BGP sessions are currently flapping, resulting in sporadic broadband connectivity. Ethernet Circuits are currently unaffected. We will update this page as more information is available. https://status.cilix.cloud/incident/725069 Sat, 20 Sep 2025 02:00:58 -0000 https://status.cilix.cloud/incident/725069#fbaafdacaef57548aecf970c37472c8c4e3ef3c5e9d8d1fcd3c9833093609340 We will be performing code upgrades on our Web Application Firewall cluster. Traffic will be moved away from nodes while upgrades are applied. Applications routing through the WAF cluster may experience up to 5 minutes of disruption. https://status.cilix.cloud/incident/717593 Sat, 06 Sep 2025 01:00:08 -0000 https://status.cilix.cloud/incident/717593#c4e31cecf7b966324adbb7c5a567087e236949f3ab25142dd097806b2f240282 We will be performing maintenance on our London 2 Border Routers on Saturday, 6 September 2025, from 0200 to 0800. A brief disruption to traffic will be seen while we drop BGP sessions to the affected border routers. https://status.cilix.cloud/incident/705964 Sat, 16 Aug 2025 00:00:00 -0000 https://status.cilix.cloud/incident/705964#4a1d49dd2061557af73b4e73f01795ec614ffb5892a70c4decb7e6e284c2c60e ##What's Happening?## On 16-Aug-25, we will be making changes to our Sentinel Web Application Firewall system to increase its request processing capacity significantly. These changes will involve adding additional compute capacity to our load-balancing and scrubbing infrastructure components. Works are scheduled to begin at 1 AM BST and are expected to be complete by 5 AM BST. ##What's the impact?## Traffic that flows through the Sentinel WAF will experience up to 4 consecutive instances of disruption, each lasting up to 5 minutes. During each instance, customer service using the WAF will be unavailable for up to 5 minutes. We have taken steps to minimise disruption where possible; however, due to the nature of the changes being made, we require all traffic to be halted while configuration is synchronised between infrastructure components. ##Updates## Updates will be provided as required. https://status.cilix.cloud/incident/604720 Sat, 28 Jun 2025 02:00:25 -0000 https://status.cilix.cloud/incident/604720#b616b2d3178b1fb8326ae4b43b13929406c13e80678fa632484f70b24c32449b ##What's Happening?  On 28-June-25, we will be performing configuration changes on the following L2TP Network Servers (LNS). These changes are to enable a number of technologies used for providing IPv6 over broadband sessions.  -lns1.lon2.as215638.net -lns2.lon2.as215638.net Any active sessions will be dropped when we apply configuration changes, the LNS' will require reloading.  These works are scheduled to begin at 3AM. ##What's the impact?  Existing broadband sessions will be terminated as the configuration changes are applied and the LNS appliance reloads. Any customers terminating on the above LNS' will experience a brief loss in internet connectivity while their session redials to an available LNS. ##Updates: We will provide updates as required on this page. If you have any queries, please get in touch.  Regards,  CilixCloud Team https://status.cilix.cloud/incident/582823 Sat, 31 May 2025 02:00:14 -0000 https://status.cilix.cloud/incident/582823#07dbb1aa0ff3d23b5a37b118973877f5366a3e479c3f2074932cfd52b83fd135 ##What's Happening?## On 31-May-25, we will be performing code upgrades on the following L2TP Network Servers (LNS). * lns1.lon2.as215638.net * lns2.lon2.as215638.net Any active sessions will be dropped before the code upgrade starts. These works are scheduled to begin at 3AM. ##What's the impact?## Existing broadband sessions will be terminated before the code upgrade begins. Any customers terminating on the above LNS' will experience a brief loss in internet connectivity while their session redials to an available LNS. https://status.cilix.cloud/incident/524353 Sat, 08 Mar 2025 03:30:44 -0000 https://status.cilix.cloud/incident/524353#f2ad1cbfa3aea980883752fd38ecdfae506ab370814d6bcfaa133e18b9eb9621 ##What's Happening?## We will migrate the SVIs used for self-service dedicated servers away from our core and onto a dedicated VC switch stack in rack A4. This is part of an ongoing project to migrate our network to a BGP-free, L2-free core, improving performance, reliability and scalability. ##What's the impact?## There will be a brief disruption in connectivity while the SVI is dropped from core switches, and brought up on the new switches. This should last a maximum of 1 minute, while our iBGP mesh reconverges. https://status.cilix.cloud/incident/523766 Sat, 08 Mar 2025 03:30:00 -0000 https://status.cilix.cloud/incident/523766#1d9e3b799c7ee5a1e3549cd09769cc2a89d6aa0d35f83b2709892b53871780da ##What's Happening?## We will be performing firmware upgrades on Cisco access Switches in racks A3 & A4 to patch a recently discovered CVE vulnerability.  These works are scheduled to take place on 08-March-2025, starting at 03:30 AM. ##What's the impact?## Switches will continue to pass traffic while the new firmware is installed. Once complete, they will reload, which will take approximately 5 minutes. During this time, network connectivity will be unavailable.  Switches will be updated sequentially, to minimise disruption. Customers who take redundant Layer 3 connectivity from us will failover to their backup uplinks automatically. We will cull BGP sessions for customers who advertise their routes to us using BGP before starting the upgrade process. This will gracefully route traffic away from the affected uplinks. https://status.cilix.cloud/incident/516796 Sat, 22 Feb 2025 07:21:00 -0000 https://status.cilix.cloud/incident/516796#8a9291b34636b245d9b1c12622fa638e7e0572eb4fd8532424887a3ba429a64e We have observed an extended period of stability, all broadband services are online. We'll update this incident with further details as we receive them. https://status.cilix.cloud/incident/516682 Sat, 22 Feb 2025 01:00:00 -0000 https://status.cilix.cloud/incident/516682#4a0b4c17c5e20f0aa03b5e977e7063585d04123e18b0ae436224f01a8aff1e1c Our power provider in London 2 will perform maintenance on one of the redundant power connections in London 2. During this time we will be operating with reduced power redundancy. https://status.cilix.cloud/incident/516796 Fri, 21 Feb 2025 15:34:00 -0000 https://status.cilix.cloud/incident/516796#a619c249d77321b21e3492bdb4e32b4090528bc11d592abd4b608cb956fab9f5 All broadband sessions are back online - We have not had confirmation that the issues on the provider's side have been resolved, as such, we are still considering broadband services at risk of further disruption. https://status.cilix.cloud/incident/516796 Fri, 21 Feb 2025 15:02:00 -0000 https://status.cilix.cloud/incident/516796#619d7b8f4dc8bf0fd1afeeaa00c31244bcff21760ab8549f36a76da8c4480769 We can see broadband sessions are now dialing via alternative paths, with the majority of sessions back online. https://status.cilix.cloud/incident/516796 Fri, 21 Feb 2025 14:44:00 -0000 https://status.cilix.cloud/incident/516796#c9510b1aad89a5662b5627cf9c12ae323498e5183074bcbdf8b8138d43b5f3ad We are aware of a network device failure in BTs POP in THW. Sessions are diverting via alternative paths, however, continued issues in the provider's THW POP are causing disruption. We are working to isolate the THW NNIs, and route L2TP traffic via alternative paths. https://status.cilix.cloud/incident/516796 Fri, 21 Feb 2025 14:40:00 -0000 https://status.cilix.cloud/incident/516796#a8df48bdec0737c1bda6fb620bc88ef41540f11e15de9a3080322c24c9fe5a18 We are aware of an issue with broadband sessions connecting via BT's Telehouse West POP. We are working with the BT to isolate the root cause. https://status.cilix.cloud/incident/489709 Sat, 11 Jan 2025 01:30:44 -0000 https://status.cilix.cloud/incident/489709#7b7b35e9a1d7c75832fca51f0e22b21b26e5eda72f85d41cbbfc9fb78717ace9 ##What's Happening? We will be making changes to our core network to extend our EVPN fabric between our London 1 and London 2 facilities. These changes will bring VXLAN tunnels between LON1 and LON2 online, enabling HA, failover, and L2 communication for resources hosted at both sites. These works are scheduled to take place on 11-Jan-2025, starting at 01:30 AM. ##What's the impact? We do not anticipate downtime or disruption during these works, however, as changes are being made to our core network, service availability is at risk until the changes are complete. ##Updates: Updates will be provided on this maintenance note as required. https://status.cilix.cloud/incident/489760 Thu, 02 Jan 2025 11:27:00 -0000 https://status.cilix.cloud/incident/489760#e7d0851844d2f2ddd7495741b4e291c9d69a5e5c1f3689dfab2ce197cc63204c BT NOC has provided the following information: _________________________________________________________ Date and Time of Incident: 02/01/2025, 10:04 AM Summary of Impact: At 10:04 AM on 02/01/2025, a brief disruption occurred, impacting all services routed through the THW POP. Leased Line services restored almost immediately. Broadband services experienced varied recovery times, depending on customer router PPP dialer frequency, with some requiring a manual reboot to re-establish connectivity. Current Status: We are actively investigating the root cause of the incident. Preliminary analysis suggests a potential hardware fault. As part of our investigation, we are consulting with our vendor, Cisco, to determine the specific nature of the issue and identify any required remedial actions. Resolution Status: Pending further investigation and vendor feedback. We apologise for any inconvenience caused and are committed to resolving this issue as quickly as possible. https://status.cilix.cloud/incident/489760 Thu, 02 Jan 2025 10:06:00 -0000 https://status.cilix.cloud/incident/489760#bdcc2f752a0c3b9c7729d5e25bd5984251b0665daca9a069f4c101ebd6e31821 All affected services are back online and experienced approximately 90 seconds of disruption before failing over to alternative infrastructure. We are investigating the issue with BT and will provide updates as required. https://status.cilix.cloud/incident/489760 Thu, 02 Jan 2025 10:04:00 -0000 https://status.cilix.cloud/incident/489760#1bd124f5b7d26ced28ca2145e0cb861ad7e84291dbf1fffbbda1c99e99b7771d We are aware of connectivity issues for BT Wholesale broadband and leased line circuits terminating in Telehouse West. We are investigating these issues and will provide updates in due course. https://status.cilix.cloud/incident/474427 Sat, 14 Dec 2024 12:00:00 -0000 https://status.cilix.cloud/incident/474427#307fec1ce2b86f8ae2848f0343129735e16d38b70fbdf312d4bd8bf2171cf79c Maintenance completed https://status.cilix.cloud/incident/474427 Sat, 14 Dec 2024 01:00:00 -0000 https://status.cilix.cloud/incident/474427#707be83adf32a8ff016318d99290d9e7257dd3ebbcdf78c52f1dc1cdb35ce270 We will perform maintenance to integrate our new LON2 hosting facility with our network POPs, and our LON1 hosting facility. As part of this work, we will bring DCIs (Data Centre Interconnects) online between each site to enable additional redundancy, resiliency, and flexibility to customer services. We do not expect there to be any disruption as a result of these works, however, as we will be making changes to our core routing equipment there is a small risk of disruption. Updates will be provided as required. https://status.cilix.cloud/incident/440763 Tue, 08 Oct 2024 14:18:00 -0000 https://status.cilix.cloud/incident/440763#02d13cf898dd84cdb81177b74d444a119176cc0f9cb58759a3c6b360de6a2d97 # What happened? ### Tuesday 08-October-24 at * 19:00: We received alerts from our monitoring systems that our primary power feed had dropped offline. No disruption to service availability at this point. * 19:20: We receive alerts that several access (feed) switches have gone offline and are unreachable internally. * 20:59: We receive alerts that the offline access switches have come back online and are carrying traffic correctly. * 21:05: We manually verify that workloads are back online and functioning as expected. # What went wrong? After our primary power feed dropped, all equipment successfully failed over to the backup power feed and was performing as expected. In an effort to bring service back online for non-power redundant customers, our colocation provider enabled a transfer switch, that connected the primary and secondary power feeds together. This in turn meant that all the previously offline equipment in the data centre began to power back online. This resulted in a voltage drop from the standard 240v, to approximately 160v. Our core, border and aggregation network equipment feature power supplies that are able to function within a 100-240v range, and as such weren't affected by the voltage drop. However, our access switches feature power supplies that are validated for 220-240v. This led to our access switches powering down automatically, which in turn caused a loss of connectivity to any services not directly connected to our core switches. Once the voltage drop condition cleared, the access switches powered back online automatically, and service availability was restored. # What changes are we making? The service disruption was caused by a failed power feed, which then resulted in out-of-range operating conditions for our access layer switches. As a result, we will be implementing changes to our access layer switches that include: * Firmware configuration changes to adjust the automatic shutdown voltage range. * Replacement power supplies, validated for a 100-240v voltage range. The above changes are scheduled to take place this Saturday and Sunday (12-13 October 2024) We will also provide further updates once we have received a full Root Cause Analysis from our colocation provider. https://status.cilix.cloud/incident/440763 Mon, 07 Oct 2024 19:45:00 -0000 https://status.cilix.cloud/incident/440763#d76da99b0fe530cede900de31ff5386c845f97a6f8c5deaecfd8850ece9ba478 We have seen a full restoration of power at DC1. We are currently manually verifying all equipment has powered on correctly. https://status.cilix.cloud/incident/440763 Mon, 07 Oct 2024 19:30:00 -0000 https://status.cilix.cloud/incident/440763#315846d2c15036a30d6adc57f691c86c988db832126c905a18b2e6b3e72f39b9 We have received an update from the data centre that the issue is related to power systems. They believe they have identified the cause of the issues, and are working on remediation. https://status.cilix.cloud/incident/440763 Mon, 07 Oct 2024 18:59:00 -0000 https://status.cilix.cloud/incident/440763#497ee5768c5a3022a7cf25b8026e2925f0bc23f614cd047c91390eb676c6890e We are aware of an issue affecting the availability of our infrastructure. https://status.cilix.cloud/incident/427496 Sat, 14 Sep 2024 04:00:00 -0000 https://status.cilix.cloud/incident/427496#581f5d60181913f27aa8c23a1449e131619a2417f2f665e1633a4780ec36477a We will perform maintenance on our network infrastructure in our London 1 location to upgrade our access layer switches to 25Gb capable equipment. As part of this, we will be re-patching hypervisor nodes into the new access switches, however, we will re-route traffic through alternate uplinks beforehand. Dedicated server customers with the prefix "lon1-a4" will be unaffected by the maintenance. We expect disruption to be minimal and will update this page with more information as required. https://status.cilix.cloud/incident/406059 Thu, 01 Aug 2024 14:00:00 -0000 https://status.cilix.cloud/incident/406059#374e776c479932d0e06af797b78d0a0ac61e4e206b444aa884f43456179c7ceb We will be deploying a software update to our self-service portal. During the update works, the portal will be offline. *Please note, that customer virtual and dedicated servers will remain online and aren't affected by these maintenance works.* https://status.cilix.cloud/incident/351449 Sat, 06 Apr 2024 06:00:00 -0000 https://status.cilix.cloud/incident/351449#c10ed78d595d3ff622d9cbe29b48fa3697260f8960fb31fcb8726c00f5b9813a As part of an ongoing project to enhance the security of our infrastructure, we will be integrating Cloudflare's Magic Transit DDoS protection into our infrastructure. As part of these works, we must deploy changes to our routing infrastructure and traffic monitoring systems. These changes will be made during a 2-hour window on the 6th of April 2024, resulting in up to 10 minutes of downtime. We will provide further updates as necessary.