Curent Status - Integrated System Monitoring & Notification Services
Planned Maintenance - Integrated System Monitoring & Notification Services
Integrated System Monitoring & Notification Services (Last 90 days)
System History - Integrated System Monitoring & Notification Services
Planned maintenance - Integrated System Monitoring & Notification Services
On March 28th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Initial Post ---
On February 28th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Initial Post ---
On January 31st, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
OIT's Zabbix service has been upgraded and returned to service.
--- Initial Post ---
On December 2nd, from 10:00 AM to 12:00 PM, CSI will be upgrading the OIT Monitoring (Zabbix) Service from 6.0 to 7.0. We expect the upgrade to take about an hour. Zabbix users should consider the service to be unavailable for monitoring hosts and delivering alerts.
Based on our testing, the upgrade to Zabbix 7.0 should not change how we currently use the service.
We will use the following communication platforms to make any additional announcements during the maintenance window.
"OIT - Monitoring" Google Chat Space
Link to OIT - Monitoring chat space
"NC State IT" Slack organization's #monitoring channel
Please feel free to reach out to us with either of the above communication platforms or via oit_monitoring@help.ncsu.edu if you have any questions or concerns.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Initial Post ---
On October 25th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Initial Post ---
On September 27th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Initial Post ---
On August 30th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
OIT's Zabbix, Iris and Oncall services have been updated and restored.
--- Initial Post ---
On July 26th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Initial Post ---
On May 31st, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Initial Post ---
On April 26th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Follow up ---
The zabbix, iris and oncall services have been updated and restored.
--- Correction ---
Maintenance on zabbix.oit.ncsu.edu and iris.oit.ncsu.edu will occur on Friday, February 23rd.
--- Initial Post ---
On February 23rd, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The monitoring and alerting services have been restored to service.
--- Initial Post ---
On January 26th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
OIT's Zabbix, Iris and Oncall services have been updated and restored to service.
--- Initial Post ---
On November 17th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Outage - Integrated System Monitoring & Notification Services
Update: 11/6/2023 10:21am All monitoring services restored
The database server that powers the OIT CSI Monitoring Infrastructure experienced a network outage at 7:50am and is currently being restarted. The Zabbix, Iris, and Oncall services are unavailable as this database is restarted.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The monitoring and alerting services have been restored to service.
--- Initial Post ---
On October 27th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The Zabbix and Iris/Oncall services have been updated and restored to service.
--- Initial Post ---
On September 29th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
Maintenance on OIT's Zabbix and Iris/Oncall is complete.
--- Initial Post ---
On August 25th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The Zabbix, Iris and Oncall services were restored to service at 9:30 AM after being updated and restarted.
--- Initial Post ---
On June 30th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Degradation - Integrated System Monitoring & Notification Services
The Zabbix service was in a degraded state from approximately 1:30 PM to 2:30 PM on May 30th, 2023. The disk utilization of the storage where the database is stored climbed to 100%. This condition persisted and by about 1:50 PM, the Zabbix service was unable to properly collect metrics from monitored hosts. As a result, Zabbix reported "Zabbix agent is unavailable" for many hosts. This caused lots of notifications to be sent to the users responsible for those hosts.
At about 2:20 PM, CSI restarted the Zabbix and database services. The services returned to normal behavior.
We are continuing to look for a root cause. So far, logs and metrics show no sign of unusual server behavior that would have triggered the event.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The Zabbix, Iris and Oncall services have been updated and rebooted.
-- Original Post ---
On May 26th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The Postgresql database for the Zabbix service has been upgraded. The service is running normally.
Please feel free to reach out via oit_unix@help.ncsu.edu or #monitoring in NC State IT Slack for any questions or concerns.
--- Intial Post ---
CSI will be upgrading the OIT Monitoring (Zabbix) Services Database backend on April 28 from 9:00 am - 11:00 am. This database version upgrade is required to support future versions of Zabbix and ensure the health of our monitoring platform. The Zabbix service will be offline during this maintenance window. That means that metrics will not be collected, the web interface, https://zabbix.oit.ncsu.edu, will not be available and notifications generated by Zabbix will not function during this maintenance.
Please feel free to reach out via oit_unix@help.ncsu.edu or #monitoring in NC State IT Slack for any questions or concerns.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The Zabbix, Iris and Oncall services have been updated and rebooted. The services are running normally.
--- Original Post ---
On January 27th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
--- Followup Post ---
The Zabbix, Iris and Oncall services have been updated and rebooted. The services are running normally.
--- Original Post ---
On September 30th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
-- Followup Post ---
Migration of Zabbix, Iris and Oncall is complete. The service is running normally.
--- Original Post ---
OIT's Zabbix monitoring and Iris/Oncall services will be offline while their virtual servers are migrated to MCNC.
Migrating the Iris/Oncall service should take under an hour.
However, the Zabbix database server is fairly large. We anticipate that it will take up to 4 to 6 hours to migrate the database server. The Zabbix monitoring service will be offline while the data is moved to the new location.
Planned maintenance - Integrated System Monitoring & Notification Services
2022-08-26
OIT's Zabbix service has been updated to version 6.0. The Zabbix, Iris and Oncall services are working normally. We recommend folks upgrade the zabbix-agents on their hosts to version 6.0 as time permits.
We will be upgrading the OIT Monitoring (Zabbix) Service on August 26th from 9:00 am - 11:00 am. Monitoring will not be available during this maintenance window.
Based on our testing, the upgrade to Zabbix 6.0 should not change how we currently use the service.
Please feel free to reach out via oit_unix@help.ncsu.edu or #monitoring in NC State IT Slack for any questions or concerns.
Planned maintenance - Integrated System Monitoring & Notification Services
We will be upgrading the OIT Monitoring (Zabbix) Services Database backend on July 29 from 9:00 am - 11:00 am. Monitoring will not be available current this maintenance window. This database version upgrade is required to support future versions of Zabbix and ensure the health of our monitoring platform.
Please feel free to reach out via oit_unix@help.ncsu.edu or #monitoring in NC State IT Slack for any questions or concerns.
Planned maintenance - Integrated System Monitoring & Notification Services
2022-06-24
Maintenance on production Zabbix and Iris servers is complete.
On June 24th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
2022-05-12
The cert has been updated. Maintenance is complete.
The TLS cert for zabbix.oit.ncsu.edu will expire on 5/29/2022. On 5/12/2022, we'll replace the cert with a new one. There will be a brief interruption for the web UI for https://zabbix.oit.ncsu.edu while the web service is restarted. This maintenance will not interrupt the collection of metrics from monitored hosts.
Planned maintenance - Integrated System Monitoring & Notification Services
Maintenance has been completed successfully.
On April 29th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
2022-03-25
Maintenance is complete.
2022-03-25
On March 25th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
2022-02-25 09:37
Maintenance is complete.
2022-02-25
On February 25th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
2022-01-28
CSI has successfully completed maintenance on OIT's Zabbix, Iris and Oncall services.
2022-01-26
On January 28th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
On December 3rd, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
Maintenance on production Zabbix and Iris/Oncall servers are complete.
Apologies for the incorrect date and time.
This maintenance will be performed on October 29th, from 9 AM to 11 AM.
On October 29th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
On August 27th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
On July 30th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services.
As a part of this upgrade, the Zabbix service will be upgrading to version 5.4.
This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
7-2-2021 09:43
Maintenance completed successfully.
On July 2nd, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.
Planned maintenance - Integrated System Monitoring & Notification Services
CSI will be deploying O/S and security patches for the Zabbix monitoring and IRIS / ONCALL notification environments. Individual parts of the environment will reboot with a potential 5 - 10 minute outage during the defined maintenance period.
Planned maintenance - Integrated System Monitoring & Notification Services
On April 30th, from 9 AM to 11 AM, CSI will be updating software packages on the production servers responsible for providing OIT's monitoring (zabbix.oit.ncsu.edu) and notification (iris.oit.ncsu.edu) services. This maintenance will involve rebooting the servers, which will cause a brief interruption of their services.
Additional announcements will be made in the #monitoring channel of the "NC State IT" Slack organization.