Curent Status - OIT Storage Services
Planned Maintenance - OIT Storage Services
OIT Storage Services (Last 90 days)
System History - OIT Storage Services
Outage - OIT Storage Services
OIT Storage Services will be performing an emergency change to the production backup environment (Data Domain) to address an issue impacting existing NAS backups.
The change will require the Data Domain system be down for a period of time where all backup/restores process will be unable to run
(Expected outage period: Dec 12th, 2024 - 2pm - 3pm Eastern Time).
All impacted teams have been notified via direct email communications.
Degradation - OIT Storage Services
At 742pm this evening Node/SP A on mcnc-ps1000-1 experienced an unexpected reboot. Due to node and path redundancy there was no end-user service disruption.
Update:
The node resumed service automatically at 750pm. An SR will be opened with our vendor.
Outage - OIT Storage Services
Updated:
All NFS shares on the storage backend are reporting online and clients are connected.
Original:
OIT's NFS service appears to have been impacted this morning around 7:39am. NFS shares provided through dc2isi00nfs or ncsudrive may need to be remounted or the server rebooted. Storage engineers are unsure of the root cause a this time.
Degradation - OIT Storage Services
Friday, Oct 6 at 6:11:14 PM, storage processor (SP) B on OIT's primary tier 1 storage array at EDC experienced an unexpected reboot.
Due to the redundant nature of the storage array and fabric there was no end-user impact. SPB came back online at 6:22:57 PM and resumed serving I/O. Engineers are currently investigating the root cause of the SP reboot and a case was automatically opened with the vendor.
Planned maintenance - OIT Storage Services
On Sunday, September 10th, between 10am and 2pm, OIT Storage engineers will upgrade our EDC fibre channel fabrics to new hardware. No outage is expected due to the redundant nature of the storage fabrics, but endpoints will experience brief periods of path degradation during this maintenance window while upstream infrastructure is migrated.
We only anticipate the migration to take 1-2 hours, but reserve the entire window in order to resolve any unforeseen issues.
Planned maintenance - OIT Storage Services
On Thursday, August 17th 2023, there will be diminished access to Research Storage while the platform (NetApp) is upgraded/patched to a more current release. Expected window for the upgrade is 2:00pm - 4:00pm, and any extension of that window due to issues encountered will be relayed to relevant teams.
Degradation - OIT Storage Services
Module replacement complete. Tier 2 frame is back in service/full redundancy.
At approximately 330pm today, a single storage processor (SPB) on the tier 2 storage frame at EDC will be brought down to replace a faulted cooling module. Due to the redundant nature of the storage frame and fabric, there should be no disruption in tier 2 block services. However there may be a brief disruption in a small subset of NAS services as those instances failover to the alternate SP.
Degradation - OIT Storage Services
Update 08-11-2023:
The batter backup unit was successfully replaced Wednesday morning 08-09-23. Tier1 storage resumed normal operations and performance.
08-08-2023
This morning at 08:36 write cache on a tier 1 storage frame was disabled due to a battery backup unit failure. We are in contact with the vendor and working to have this replaced ASAP. Due to multiple redundant paths no outages have been/are expected, but there may be brief periods of performance degredation until the BBU is replaced.
Outage - OIT Storage Services
On Monday, July 31st 2023, there will be diminished access to Research Storage while the platform (NetApp) is upgraded/patched to address issues related to Microsoft CVE-2022-38023. Expected window for the upgrade is 2:00pm - 4:00pm, and any extension of that window due to issues encountered will be relayed to relevant teams.
Planned maintenance - OIT Storage Services
An emergency expansion of disk space is required for the production Data Domain backup units, which will require all backup/restore jobs to cease during the installation (Outage period: April 10th - 10:00am to 3:00pm Eastern time)
All impacted teams have been notified via sysnews and direct email communications.
Planned maintenance - OIT Storage Services
Data center project to impact campus IT services Jan. 2
As part of the university’s data center migration project, the Office of Information Technology (OIT) will move network file shares to a new data center location on Monday, Jan. 2, which is a university-observed holiday.
As a result, all OIT-provided department file shares, user home drives and many other campus IT services will be unavailable from 5 a.m. to 2 p.m. on Jan. 2.
Impacted campus services include but are not limited to:
- OIT Network File services
- Advance System
- Financial System
- Human Resources System, including Wolftime and Time Clocks
- MyPack Portal
- OnBase Document Management System
- Oracle Identity Management including Self-Service Password Management
- RedHat Network Satellite service
- SAS Visual Analytics and Business Intelligence Platforms
- Student Information System
- System Access Request (SAR)
- Wolfpack Reporting System (WRS)
If you have questions about this service outage:
- Contact the NC State Help Desk via the NC State IT Service Portal or call 919.515.HELP (4357).
Planned maintenance - OIT Storage Services
This notice is for all users of the SMB NAS services provided by Isilon for group and home drive shares.
On Sunday December 4th, 2022 between 7:00am and 9:00am, the OIT-PCS team will be performing work on the DC2 Isilon NAS cluster to address an ongoing issue affecting overall cluster health.
As part of this work, the SMB share service will have a brief disruption, which should be transparent to most users (NFS services will be unaffected).
Outage - OIT Storage Services
OIT/Comtech will be making a change to the OIT Primary NAS services environment to address an ongoing issue with file transfers. The network connectivity to this environment in DC2 will be briefly interrupted(sub 1 mintue) to make this change.
Affects: OIT Primary NAS services -- dc2isi00smb/NCSUdrive, dc2isi00nfs, dc2isi00erp
Degradation - OIT Storage Services
Update 9/12 11:23: Issue resolved. Root cause was suspected to be related to a large number of disk errors for a single disk and related disk firmware. Vendor recommendation was to apply firmware disk updates. Firmware updates have been applied.
At approximately 7:40pm this evening, a storage processor (SPB) rebooted on our DC2 Unity 650F array. This array is the primary source of Tier1 storage for the DC2 VMware environment. Due to the presence of a secondary storage processor (SPA), there should not have been any noticed enduser impact.
Engineers are currently investigating the root cause with the vendor.
Degradation - OIT Storage Services
A faulty memory module was discovered after a scheduled SP (storage processor) reboot on the primary DC2 tier 1 storage frame.
Beginning at 830am Sunday, July 10, OIT staff will begin working with the vendor to replace this module. This hardware replacement will require a shutdown of SP B, during which time all tier 1 storage will automatically failover to SP A. No end-user impact is expected, but during this period all applications served by this tier 1 frame will be served by a single SP.
UPDATE (09:41:24): Memory module replaced. Both SPs are back in full service.
Degradation - OIT Storage Services
Monday, May 17 2021 04:49--
The dc2-u480f-0 storage frame experienced an unexpected storage processor (SPA) reboot.
All services immediately failed over to the secondary storage processor (SPB). No end-users experienced any outage during the reboot cycle and SPA was up and running bringing the frame back to full redundancy within 15 minutes.
OIT storage team staff are continuing to investigate.
Planned maintenance - OIT Storage Services
OIT Storage providing NCSU drive (B:) and departmental file space will experience outages during the upcoming extended maintenance weekend due to system upgrades and configuration changes.