CMU WiFi struggles as semester begins
On the first day of classes, David Rudo, a computer science first-year, had trouble connecting to CMU-SECURE WiFi, the preferred wireless network for students and faculty on campus. The network worked well during Orientation Week and when he was in his dorm, but whenever he went to large lecture halls or buildings on campus and tried to connect to the internet, the connection was either too slow or would not work at all.
“I pretty much ran off of a hotspot for a good week and a half before the WiFi started to work again,” Rudo said. Other students also raised concerns about the various issues they were having, including trouble connecting to the wireless network, slowed-down connections, and not being able to access the internet despite being connected to the WiFi.
On Sept. 8, Vice President for Information Technology and Chief Information Officer Stan Waddell sent a campus-wide email assuring that Computing Services is “aware of the recent wireless network issues” and “working closely with our network vendor” to fix the issues. Aruba Networks, a Hewlett Packard Enterprise company, has been Carnegie Mellon’s main campus wireless vendor for over a dozen years. The software issue Aruba found was affecting the wireless network at Carnegie Mellon and their other network customers.
In an email response to questions from The Tartan, Waddell explained that when this software issue would trigger, different small groups of devices across all of the main campus networks such as CMU-DEVICE, CMU, and CMU-GUEST, would temporarily experience anything between a successful connection with poor performance to being unable to connect to the wireless network at all. He claimed, however, the majority of devices would have noticed no impact. He explained further that Computing Services, per their longstanding standard operating procedures, had evaluated all open vendor software issues and available software versions prior to the beginning of the semester and chose the most recent, stable, and tested version of Aruba Networks’ software. He said it wasn’t until Sept. 1 that Aruba recognized the software issue that impacted Carnegie Mellon and their other large network customers.
Waddell shared the full vendor report with The Tartan, which revealed that “WiFi device connectivity failures are experienced only during high bursts of client roaming events” and that such events result in high CPU utilization by the Station Management (STM) process in the Mobility Controller(s). The root cause of this issue, the report states, has been traced to code changes made to resolve a bug. Aruba developed emergency patches to remove the code changes made as part of the bug fix.
In the email sent out on Sept. 8, Waddell announced that updates on the situation will be posted on a special wireless network status page until the issue is resolved. The page says that a workaround applied on Sept. 3 was unsuccessful, but another applied on Sept. 7 improved wireless network stability. According to the status page, the network has remained stable since Sept. 9.
The page also revealed multiple scheduled network maintenance works and related outages in the past couple of weeks. Waddell explained that Computing Services has a change moratorium in the week before and the first week of classes to promote technological stability. However, as they worked with Aruba to fix the software issue, they tested and implemented numerous solutions to determine which would help address the issue. The maintenance works listed on the status page were, in large part, Computing Services “removing these numerous workarounds in a controlled manner.”
The wireless network status page also allows for people to report any issues with the WiFi. Waddell said there have been no indicators of any wireless problems related to the software issue since the fix was applied. He also assured that Computing Services is continuously monitoring the health of the wireless network and reviewing all inbound issue reports.