As of 12:20 PM, the issue has been resolved. The root cause was identified as a DTMF mismatch resulting from an unannounced carrier change.

The UC Team has received reports that calls to the campus main line, 5-2011, are failing. We are investigating. We will provide an update once we have more info. Thank you. 

This issue has been resolved. Thank you. 

The issue affecting off-campus calls appears to have been resolved in collaboration with our vendor. We have not received any new reports of problems and will continue to monitor the situation closely.

UPDATE 8:30AM 8/19/2024

Multiple issues have been reported again with NC State phone lines making and receiving calls from non-campus phone numbers. Phone calls from and to non-campus numbers may face severe audio issues or not reach at all.  NC State's phone service provider is working with the Local Carrier to repair services again and to isolate the cause. 

UPDATE 10:30PM 8/18/2024

Our service provider has informed us the issue should be resolved. The UC Team has made several test calls and all connected successfully with normal audio. We will continue to test throughout the day ensure the issue is fully resolved. 

UPDATE 7:25PM 8/17/2024

NCSU's phone service provider has informed the UC Team that our trouble is related to a larger outage that is affecting the entire southern United States. No ETR at this time. We will continue to provide updates as we receive them. 

UPDATE 4:00PM 8/17/2024

NC State's phone service provider is working with the Local Carrier to repair services. At this time, the Local Carrier is isolating the trouble further to determine necessary actions to resolve the issue. Additional updates will be provided until service is restored.

_____________________________

ORIGINAL: 2:00PM 8/17/2024

There is currently an ongoing issue with NC State phone lines making and receiving calls from non-campus phone numbers. Phone calls from and to non-campus numbers may face severe audio issues or not reach at all.  The ComTech UC team is aware of the issue and it appears to be caused by an issue with the phone carrier. The Comtech UC team is opening a call with that carrier to investigate the issue further. 

Calls between two on-campus numbers appear to work as normal. 

Services have been restored at this time. Windstream had a fiber outage on both main and redundant lines causing the static issue that was being reported. Wind stream has resolved our ticket and is showing that service is up and operational. 

Final Update: ComTech Senior Engineer for Security was able to isolate root cause and mitigate.  All Off Campus VPN phones are showing registered within the Call Manager server.  If there are any further questions or concerns, please reach back to the NOC at https://ncsu.service-now.com/sp?id=index.

Update: ComTech Senior Engineers continue to troubleshot towards root cause.  Sites are able to communicate via data (wireless, email, etc.), but phone service remains down.  More details to follow.

ComTech is aware of four Off-Campus VPN sites whose phone communications are currently not working.  The sites are:  The Japan Center, The Belk Center, NCSU Trio, and Crop & Soil Sciences - Pest Control. Senior Engineers have been engaged since 10:00 this morning and continue troubleshooting efforts.  More details to follow.

NC State Call centers and menu-driven call handlers and options are currently not accepting menu selections from off-campus callers. We have opened an incident with our carrier.

Comtech will be performing maintenance on a circuit that provides connectivity for off campus phone calls.  It is possible that an active call could be dropped as the circuit is down.  There is a redundant link so all new calls during this maintenance will roll to the second link.

Campus phones will briefly unregister from their primary server and re-register once the service restart is complete.  Active phone calls are not impacted. If there are any issues please call the NOC at 57099 and select option 3.

Update:
Root cause was a configuration mismatch between call manager and some devices.  Mismatch has been resolved and some phones are still showing rejected and a factory reset will be needed.  For any assistance in performing the instructions below, please call the TELE group at 57099 Option 3.

To resolve the phone registration issue factory resets should be performed on phones showing 'Rejected' on their displays.

Please follow the instructions below for your model phone.

Phones: Factory Reset for 6900, 7800, and 8831 Series
https://ncsu.service-now.com/kb_view.do?sysparm_article=KB0016056

Phones: 7900 Series Factory Reset
https://ncsu.service-now.com/kb_view.do?sysparm_article=KB0016046

Carrier has resolved the issue and we believe the issue is resolved (Updated: 5/17 @ 1:10pm)

Deleting expiring call manager-trust and CAPF-trust certificates across all call manager servers. Restarting of services will cause all campus phones to momentarily unregister then re-register with the system.