System Status

Curent Status - Secured Networks

This service is not reporting an issue.

Planned Maintenance - Secured Networks

No system maintenance is planned over the next 30 days

Secured Networks (Last 90 days)

Color bar indicators flow left to right, from oldest to most recent status

System History - Secured Networks

From most recent to oldest, this list shows all outages, degradations and planned maintenance for this service

Degradation - Secured Networks

Started 2025-02-19 08:07:59, Duration 2 Hours 1 Minute
Issues with some campus connectivity
View Details
Task Number: CHG0036023

Update 2/19/25 10:05AM

We have determined the issue was related to communications between backend services. Connectivity to shared drives and login to systems should be resolved. Some logins may still take a long time as the services are restored. If you continue to have issues, please contact the Help Desk at help@ncsu.edu or https://help.ncsu.edu

 

***********************

Update 2/19/25 9:24AM

OIT Engineers are still working on this issue. Will update when we have confirmed all issues are mitigated. 

If customers are encountering issues logging in, it may take over 10 minutes until the issue is resolved.

 

***********************

ComTech is aware of issues connecting to shared drives, remote systems, and other campus resources. Our engineers are currently working with other groups in OIT to determine root cause and scope of issue. This post will be updated when we have more information.

Planned maintenance - Secured Networks

Started 2025-01-24 10:48:26, Duration 1 Day 13 Hours
Software Upgrade
View Details
Task Number: CHG0036313

Saturday, February 25th starting at 9:00 AM ComTech will upgrade the FortiGates software to address CVE-2024-55591.  

Outage - Secured Networks

Started 2024-12-06 21:21:15, Duration 2 Days
VPN Outage
View Details

This evening campus VPN experienced an outage due to authentication attempts on the VPN Temp and Vendor groups.  This caused a large number of radius attempts on those two groups.  To mitigate the issue we implemented a workaround to stop the radius request.  ComTech will continue to monitor the VPN headend throughout the weekend.

Planned maintenance - Secured Networks

Started 2024-10-30 22:00:00, Duration 1 Hour 30 Minutes
Migration of message and advance compute
View Details
Task Number: CHG0036023

We will be migrating two contexts off of the Cisco firewall over to the Palo Alto firewall.

Planned maintenance - Secured Networks

Started 2024-10-22 22:00:00, Duration 1 Hour 30 Minutes
Data Center Firewall Context Migration
View Details
Task Number: CHG0036023

ComTech is migrating the management context and this transition will cause a 5 to 10 minute outage for this service.

Planned maintenance - Secured Networks

Started 2024-06-25 15:34:14, Duration 1 Day 4 Minutes
Palo Alto 5220 Upgrade
View Details
Task Number: CHG0035489

Tonight starting at 6:00 PM ComTech will be upgrading the PanOS software on the remote PA-5220s.  This upgrade impacts our LSVPN sites and B2B(non-cloud) IPsec tunnels.  

Outage - Secured Networks

Started 2024-03-02 21:39:28, Duration 2 Hours 50 Minutes
Palo Alto software upgrad
View Details
Task Number: CHG0034869

Saturday, March 2nd at 10:00 PM Comtech will be performing software upgrades on the Palo Alto firewalls.  

Planned maintenance - Secured Networks

Started 2024-02-14 12:00:00, Duration 4 Days 1 Hour
Fortigate Firmware Upgrade
View Details
Task Number: CHG0034757

Sunday, February 18th at 7:00 am ComTech will be upgrading the firmware on the Fortigate 1200D firewalls.  

Planned maintenance - Secured Networks

Started 2023-12-19 22:00:00, Duration 1 Hour 30 Minutes
ASA Failover
View Details
Task Number: CHG0034545

Tonight ComTech will be re-enabling HA on the SMDF AnyConnect Firepower.  

Outage - Secured Networks

Started 2023-07-16 07:00:18, Duration 2 Hours
Remote Site VPN Outage
View Details
Task Number: CHG0033677

Comtech will be migrating the routing for the "Site-to-Site" VPN service used to connect remote sites to the NCSU network.  During this migration, the service will be offline.  We do not anticipate the outage to be more than 1-2 minutes.

Outage - Secured Networks

Started 2023-07-08 09:00:00, Duration 2 Hours
Data Center Firewall Upgrade
View Details
Task Number: CHG0033675

As part of the July EMW Comtech will be upgrading the software on the DC, DC1, and DC2 ASASMs.

Planned maintenance - Secured Networks

Started 2023-06-16 08:00:00, Duration 14 Hours
ASA Software Upgrade
View Details
Task Number: CHG0033560

Tonight at 7 PM Comtech will be upgrading the ASA software on the AnyConnect ASAs.

Outage - Secured Networks

Started 2023-05-31 22:00:00, Duration 16 Hours 57 Minutes
VPN Client Outage
View Details
Task Number: CHG0033503

Update 6/1/23 14:57PM

VPN Services have been re-established.  A modification to the firewall was made to allow connectivity to the Cisco License server.  For further questions, please go to this link: https://ncsu.service-now.com/sp?id=index

**********************

Update 6/1/23 14:20PM

ComTech Senior Engineers have discovered the root cause; the Cisco Licensing server is not responding to our request to validate our license. Cisco Engineers are continuing to work towards resolution.  No estimated time of repair is available at this time. When conditions have next changed, we will provide another update at that time.

Update 6/1/23 11:49AM

ComTech Senior Engineers are continuing to work with Cisco towards resolution.  No estimated time of repair is available at this time. When conditions have next changed, we will provide another update at that time.

**********************

Update 6/1/23 10:07AM

ComTech Senior Engineers are continuing to work with Cisco towards resolution. A severity 1 Cisco case has been opened. No estimated time of repair is available at this time. More information will be updated here when it becomes available.

**********************

Update 6/1/23 9:08AM

ComTech Senior Engineers have dispatched to site; continuing to work with Cisco towards resolution. A severity 1 Cisco case has been opened.  More information will be updated here when it becomes available.

**********************

update 6/1/23 8:18AM

ComTech senior engineers continue to work with Cisco for resolution.

**********************

After a routine firmware upgrade, ComTech is investigating issues with the Cisco Anyconnect VPN Client. We are receiving reports of customers unable to log in. Engineers are currently working with the vendor on a resolution and will update when more information is available. 

Planned maintenance - Secured Networks

Started 2023-05-31 01:30:00, Duration 1 Day 13 Hours 30 Minutes
ASA Software Upgrade
View Details
Task Number: CHG0033503

Tonight at 10:00 PM ComTech will be performing a software upgrading the Cisco ASA Firepowers.  A reboot is required as part of the upgrade, as a result of the reboot AnyConnect clients may disconnect.  In the event, this happens AnyConnect client connection will automatically reconnect.

Planned maintenance - Secured Networks

Started 2023-01-26 00:00:00, Duration 2 Hours
Palo Alto firewall software downgrade
View Details
Task Number: CHG0033045

On January 26th at 0000 (zero hundred hours) ComTech will be downgrading the PA-7050 to PanOS version 9.1.15-H2.  The HA clustering reduces the impact.  As part of the failover process, some traffic may be impacted as connections move from the active to the passive firewall.   

Planned maintenance - Secured Networks

Started 2022-12-06 16:30:06, Duration 1 Day 15 Hours 29 Minutes
Zone Protection Profile
View Details
Task Number: CHG0032852

In order to prevent potential DDoS attacks a zone protection profile will be applied to the GRT-Untrust zone. 

Planned maintenance - Secured Networks

Started 2022-06-13 07:00:00, Duration 7 Days
MPLS Firewall software upgrade
View Details
Task Number: CHG0032127

Sunday, June 19th at 7:00 AM ComTech will be upgrading the MPLS Palo Alto firewall to PanOS version 9.1.13-h3.  

Planned maintenance - Secured Networks

Started 2022-03-02 13:00:00, Duration 1 Hour
Cisco Anyconnect Headend Client Upgrade
View Details
Task Number: CHG0031805

On 3/2/2022 at 1:00 PM ComTech will be upgrading the AnyConnect VPN headend client for Windows to version 4.10.03104. The client upgrade will start automatically after you successfully authenticate to vpn.ncsu.edu.

Planned maintenance - Secured Networks

Started 2022-02-15 17:01:00, Duration 2 Hours 59 Minutes
PA-220 Software Update
View Details
Task Number: CHG0031772

The Comtech Security team will be upgrading the PA-220s at 4000 Chi Road, 4116 Reedy Creek Rd, 3920 Dr. Bill Gilbert Way and 2319 1/2 Champion Court to the approved version 9.1.11 on Tuesday, Feb 15th after business hours. The upgrade will take approximately 2 hours and then normal operations will resume. We will be monitoring during the upgrade for any issues in order to address them as quickly as possible.

 

 

Planned maintenance - Secured Networks

Started 2022-02-08 17:01:00, Duration 5 Days 6 Hours 58 Minutes
Remote VPN sites PA-220 upgrade
View Details
Task Number: CHG0031756

On 2/8/22, from 5:01pm to 8pm,  ComTech will be performing an upgrade to the PA-220s at the following remote vpn sites:
ASG -Gregg Offsite Storage
Crop & Soil Sciences (Building 38)/Pest Control
Solar House
WestChase 4011

and at Japan Center (Spring Hill House) on Sunday 2/13

This is an after business hours planned upgrade and the site will be offline during the upgrade. The site contacts have been notified and Comtech will monitor until upgrade is complete.

Planned maintenance - Secured Networks

Started 2022-02-03 17:01:00, Duration 1 Hour 59 Minutes
HBB PA-220 upgrade
View Details
Task Number: CHG0031714

On 2/3/22, from 5:01pm to 7pm,  ComTech will be performing an upgrade to the PA-220 at HBB 706 Hillsborough St. Suite 100 Raleigh, NC 27603. This is an after business hours planned upgrade and the site will be offline during the upgrade. The site contact has been notified and Comtech will monitor until upgrade is complete.

Planned maintenance - Secured Networks

Started 2022-02-02 17:01:00, Duration 1 Hour 59 Minutes
NCSU TRIO PA-220 upgrade
View Details
Task Number: CHG0031713

On 2/2/22, from 5:01pm to 7pm,  ComTech will be performing an upgrade to the PA-220 at NCSU Trio 407 Gorman St Raleigh .  This is an after business hours planned upgrade and the site will be offline during the upgrade. The site contacts have been notified and Comtech will monitor until upgrade is complete.

Outage - Secured Networks

Started 2020-12-14 09:00:00, Duration 3 Hours 6 Minutes
Connectivity issues to Google Services
View Details
Task Number: INC3671373

On Monday Dec 14th Comtech received reports of connectivity issues to Google services from secured networks.  This coincided with the time Google was also experiencing issues so initial investigation believed it to be related to Google issues.  Further investigation found an issue with a NAT policy affecting traffic destined to Google IP ranges. This was identified at roughly at 12pm on Dec 14th and corrected.  If anyone experiences any other issues connecting to Google services from the secured 10.x networks please contact the OIT NOC.