Have you ever wondered what goes on in the IT department at Boise State? Max Davis-Johnson, CIO Office of Information Technology, introduces this short video that takes you through a day in the life of the IT department at Boise State.
>> MAX: Hi, this is Max Davis-Johnson, Boise State University Office of Information Technology. I’m going to talk a little about life in OIT.
Higher Ed IT is unique. And it’s not any one thing we do, it’s just the variety of things that we do.
We have transactional systems. We provide servers and storage. We provide desktop systems. We manage applications. We provide this huge open network that needs to communicate all across country and provide wireless.
We provide help desk. We provide classroom technology. There’s just a lot of things that we do in our life in OIT.
But we’re driven by three core principles, directives and these are: one, we want to keep the trains running on time.
And what I mean “keeping the trains running on time,” systems need to be available, the networks need to be available, data needs to flow from system to system, upgrades to all the systems need to be made.
If we’re doing our job right there, you don’t even notice. And that’s the way it should be.
Another core driver is we want to be laying new track. Technology is always changing. We have to examine it and figure out what direction we want to take the track. We always have to be working to improve how our trains run.
And with the new things coming out we have to be able to go new routes.
And then the last thing is we want to create raving fans. And what I mean by that–you’re excited, you call us, we get you the help you need, we get you the support you need, you like what we do, we help you get your work done and get your job done.
So, I want you to just watch the rest of this video and it gives you a sense for what we do in a day in OIT.
>> NARRATOR: Today, over 17,000 students, faculty and staff will sign in to the Boise State University Wi-Fi network, a complex system built and maintained by our network and accounts teams who installed and manage access to over 1,000 Wi-Fi access points on campus. And our cybersecurity team ensures our network is secure and our data is safe.
Close to 15,000 people will sign in to myBoiseState, a custom web application that provides access to our email, student data, HR, learning management and other systems. myBoiseState was developed here on campus by our communications and web application development teams.
Our web application development team, working with project managers and business analysts in our project management office, build and maintain other custom applications like Major Finder that connects students with the best major based on their interests, aptitudes and career goals.
Our project management office also coordinates the work of our core systems development teams, who enhance and refine our HR, student data and learning management systems.
All of these applications run in the Boise State private and hybrid clouds designed, monitored and maintained by our system engineers and application administrator teams to make sure all of our servers, networks and architecture is up to date and backed up every night.
Our executive leadership team works with key University stakeholders to resolve the issues of today and make plans to address the needs of tomorrow.
The communications team makes sure the Boise State community is aware of maintenance outages and upcoming developments. This team also provides technology training and maintains the university website.
Throughout the day, just over 1,000 people will sign in to public computers maintained by the Customer Care hardware team. Faculty will sign in to computers in more than 100 classrooms and utilize technology designed for teaching and learning maintained by our Learning Technology Solutions team. Who, in addition to maintaining technologies like classroom capture video recording, also explore technologies like our new student response system. This allows instructors to quiz students, in real time in the classroom, all through students’ smartphones or computers.
In three primary locations on campus, the Help Desk, a blended team of professional staff and student workers, will assist around 200 people with a variety of technology solutions from troubleshooting account issues to installing software to teaching basic features of Microsoft Office and Google Apps.
Outside of class, students and instructors will generate over 30,000 sessions in Blackboard, our learning management system, where they upload assignments, interact with one another and review classroom capture videos on their computers and mobile devices.
Working with our R2 high performance computing cluster, researchers will process massive amounts of data and calculations that inform scientific discovery, bench research papers, and state-level decision makers.
And tomorrow morning, we’ll be here early to do it all again.
We are the Office of Information Technology at Boise State University.
The Sixth Annual Bronco Appathon returns to Boise State March 10 – 12, 2017.
Open to all registered Boise State students, Bronco Appathon is a weekend marathon of coding to earn cash prizes and awards. Free food and beverages are provided to participants and raffles will be held throughout the weekend.
Students form teams of up to four members and begin developing an app on Friday evening, then present their work to a panel of judges on Sunday afternoon.
Cash prizes of up to $500 per team member are available, with additional awards for Best Design and Best Novice App.
Registration is open to current full-time and part-time Boise State students through Thursday, March 2 and is free of charge.
For more information about Bronco Appathon 6, visit oit.boisestate.edu/appathon.
Last week TechSmith released a new feature to our Boise State Relay Library: folders!
You can now create folders and subfolders to organize your videos and media content in TechSmith Relay Library. You can also share a link to a folder which makes it easy to share a series of videos. More functionality will be added in the future.
Visit the TechSmith Relay Library Folder page on the Boise State Video website for more information.
If you have additional questions about TechSmith Relay Library, contact the OIT Learning Technology Solutions Team at firstname.lastname@example.org
The Office of Information Technology’s Research Computing Support Department will host a Coffee and Donuts for Researchers Forum on Thursday, Feb. 2, in Albertsons Library Room 201C. The room is on the second floor through the College of Innovation and Design.
The Boise State research community is invited to mingle from 9-10 a.m. and attend a special presentation from 10-11 a.m. titled “Making Sharing Data Automagic with Globus.”
Globus subscription services allows researchers to share and transfer data, no matter how big, with anyone from just about anywhere. Watch a demo and to see how you can access Globus. Attendees also will be able to ask questions about the department’s new compute cluster, R2, which is now up and running.
For more information, visit rcs.boisestate.edu.
The Boise State community is invited to attend the First Annual 2017 Process Improvement Symposium on Thursday, January 26 in the Jordan D Ballroom, Student Union Building, 10:00 am – 12:00 pm.
Boise State’s Department of Systems and Process Improvement will celebrate individual and department nominees and winners who have demonstrated innovation in University business processes.
Nominees displayed excellence in leading change, eliminating redundant systems, reduced the University’s carbon footprint, and improving efficiency across campus.
To view nominees for outstanding individuals and departments, visit the Systems and Process Improvement website.
For additional information, please contact email@example.com.
The Office of Information Technology will perform critical network maintenance between noon and 8:00 pm on Friday, January 6.
Potential impacts to the campus community are expected to be few, brief, and intermittent. Internet access, telephone calls to or from off-campus, or VPN (virtual private networking) access may be momentarily interrupted as equipment is installed.
This necessary maintenance is due to an equipment failure that occurred last month.
If you have questions, please contact the Help Desk at (208) 426-4357, or email firstname.lastname@example.org.
The Help Desk at all Zone locations will have reduced hours Dec. 17, 2016 – Jan. 8, 2017. Regular hours for all Zone locations resume Monday, Jan. 9, 2017.
The Zone at the Interactive Learning Center
- Friday, Dec. 23, 2016, through Monday, Jan. 2, 2017: CLOSED
- Tuesday, Jan. 3 – Friday, Jan. 6: 7 a.m. – 5 p.m.
- Saturday, Jan. 7 and Sunday, Jan. 8: 10 a.m. – 5 p.m.
The Zones at MBEB, MP, SUB and Keiser Hall
- Saturday, Dec. 17 – Sunday, Jan. 8: CLOSED
Equipment checkout from The Zones will not be available Dec. 15 through Jan. 2.
“We’re over 95% virtualized, and that’s unheard of in higher ed,” Max Davis-Johnson, Chief Information Officer at Boise State University, tells me over the blues music and mechanical sounds of seasonal lattes being made at Starbucks in Albertsons Library. “We run everything in this: PeopleSoft, Blackboard, all our applications, everything we do here on campus runs in this environment. It’s really cool. This is our version of the cloud.”
So, what is the cloud?
Commercials tell us, “it’s in the cloud,” as if the cloud is some specific place or thing–it’s not. Generally, “the cloud,” is simply software and services that run over the Internet rather than on our computers. On the backend, the “cloud” is a system of networks and servers designed to handle specific tasks. And, there three basic types of clouds: public, private and hybrid.
Watching movies on Netflix or Amazon are examples of public cloud services. When we use Gmail, Google Drive and G Suite (formerly known as Google Apps), we’re working in Google’s private cloud.
Unlike public clouds that anyone can access, a private cloud is limited to one organization. If you have rights and access to a private cloud, you have access to the networking, virtualized storage and computing power of that cloud. The Office of Information Technology at Boise State uses this model to provide PeopleSoft, Blackboard and WordPress in what’s called a software as a service (SAAS) model. The software runs in our virtualized environment and is delivered over the network. We don’t have to install the software on our computers, it just runs in a browser. You must have a Boise State account to access the Boise State private cloud.
What’s really exciting is that we’ve implemented a hybrid cloud model at Boise State. We utilize the private cloud model for the systems described above but we’ve also integrated with other private clouds, primarily Google and Amazon. Gmail, Google Drive and G Suite applications run in Google’s cloud. We leverage Amazon Web Services (AWS) to run some features of myBoiseState. Of course, as students, staff and faculty, we sign in and seamlessly navigate among the services provided among these private clouds.
Tory Jamison, Associate Director for Cloud Services and Infrastructure, tells me that this integration of several private clouds is the key to our hybrid cloud infrastructure, “If we took a little time we could completely obfuscate the fact that your email was coming from Google if we wanted to make the effort.” He goes on to describe that “We provide servers and storage in a highly scalable way and we can leverage resources in the public cloud or on premises from our private cloud in the familiar way.” This is how we’re able to provide storage and computing power to researchers, colleges and departments around campus, “What we have, which is very unique, is infrastructure as a service (IAAS),” he tells me.
“We have to give it away for free”
Tory had the idea to virtualize the entire university onto a private cloud almost four years ago. Prior to that, all of our colleges and auxiliaries were still using the obsolete “rack and stack” model where every server was a physical server racked one on top of the other. One application on one physical rack mounted server is no longer the way it’s done. Although OIT managed about 90% of the network at the time, we only managed about 50% of the servers on campus. This meant most Colleges and auxiliaries were purchasing and running their own servers and data centers which meant the University wasn’t leveraging economies of scale. “We wanted to change all that,” Tory says.
OIT had already attempted the chargeback models for departments and researchers to utilize our network, storage and compute infrastructures, but, Tory says, “No one came.” That’s when he had the idea to move toward a converged architecture–a virtual private cloud that could scale in both compute and storage for years to come.
“We shouldn’t be innovative in everything we do, there just isn’t enough time in the day,” Tory says emphatically, “There are commodity concepts out there and we always look to leverage those efficiencies first. This allows us to focus our attention on innovation in areas that complement the direct education of students and furthering Boise State research interests. That’s why we’re here!”
Tory’s idea to converge and fully virtualize relied on a simple premise though and he told Max, “the only way this is going to work is if we give it away for free.”
“Of course it’s not actually free, it’s more like no cost to the colleges, but we’ll take those donations if you have somebody in mind,” Tory says. The idea is that OIT provides cloud services to anyone on campus and the University pays for these services once–through the OIT budget. “We’re too big to fail,” Tory explains, “If we run out of compute, or run out of storage, everybody runs out of compute and storage but it hasn’t happened yet, not even close.”
And it’s working. In the last three and one half years, Tory’s team took out nine data centers around the University and converged these into seven racks of virtualized platform in our data center here on campus. “Those racks run the entire University,” he says smiling. “When we absorbed all these colleges, not a single person was let go. We absorbed all those people. Somebody’s still needed to run the individual applications. We don’t have enough people to run 1,400 different applications! Using the economies of the cloud model, we want to provide the colleges and researchers with all the compute and storage resources they need so that they can spend their money on educating students or furthering their research.”
We also have a disaster recovery data center about five miles from campus, not in the Boise River floodplain. Everything on campus is backed up there every night. All of our tier 1 applications and data, like Blackboard and PeopleSoft, are backed up every 15 minutes. The idea is that if something happens to our primary data center, the disaster recovery center would take over. To emphasize how this works, Tory tells me, “It could be so smooth that you wouldn’t know our primary data center burned to the ground unless you were standing there watching it smoke.”
From a security standpoint, the data center is behind the University firewall so it’s protected. The entire Boise State cloud is monitored and managed by professional staff at Boise State.
Scalable, flexible, elastic…and fiscally responsible
“The thing about virtualization is that it’s scalable, flexible, elastic,” Max says. “In higher ed we are so diverse. Any one thing we do is not necessarily diverse, but then you look at everything we do, it’s pretty unique.” Tory puts some numbers to this for us, “We have about 1,400 virtual machines running around 1,200 different servers and applications.”
With over 95% of the campus virtualized, Boise State leads the way in higher education. “I haven’t talked to a single university that even comes close,” Tory says. “And when I go speak about this at conferences, representatives from other universities will come up to me and say, ‘Holy cow, we’ve been trying to do that for 20 years!’”
“[Virtualization] is cost effective because we buy at scale,” Max says. Tory explains that our hardware gets replaced every five years and that we’re using a new model he calls just-in-time purchasing, “Because the longer you wait [to purchase hardware], the more technology progresses and the less expensive it gets.”
“It’s far less expensive for us to do this ourselves. We couldn’t put 1,400 machines and 3.5 Petabytes of data in the public cloud,” Tory says. “It would cost something like $400,000 per month. We run the whole operation on $1.1 million per year.”
Running our own cloud saves us $3.5 million–annually.
We have storage and computing at your favorite price: free
Max explains that running our own cloud, “allows us, in the blink of an eye, when a faculty researcher says, ‘I need 10TB of storage,’ 30 minutes later it’s done.” “It takes me longer to ask who needs access to the data than it takes for me to set up the space,” Tory jokes. “Someone recently asked me for 125TB and we were like, ‘We got it.’”
“That’s what being in the cloud is all about–being able to provide stuff from a pool of infrastructure, to anybody, on demand,” Tory says.
“And, we have the R2 cluster coming online soon,” Max tells me. The R2 cluster will dramatically increase our computational power for research computing at Boise State.
Contact the Office of Research Computing for more information, or to request servers for storage or computational processing (or whatever you need), at email@example.com.
Finally, you might want to check out the “Mad Max’s Crazy House of Research Computing” advertisement. Max wants to sell you some servers at your favorite price: free!
We routinely run automated processes to ensure the right people have the right computer system roles and access at Boise State. Early this morning, one of those processes went awry.
A script that tells one of processes what to do was not interpreted correctly, for reasons we’re still trying to figure out. Whatever the cause, this resulted in a worst-case scenario that detached University accounts from their associated roles.
Put simply, people’s accounts lost rights to systems and services.
This incident triggered automated email notifications to send to all active Boise State students to inform them that, because their University status was erroneously defined as suspended, they had 30 days before their student email access was cut off.
We restored account rights and access within about 90 minutes of figuring out what happened.
But we couldn’t prevent students from seeing an email saying they were, essentially, no longer students. The confusion and stress this caused for many students was, understandably, heightened by the fact that the end of the semester is naturally a source of high anxiety. We just happened to compound it.
We sent 50,000 follow-up emails to our University community apologizing for the errors, and we mean it when we say we’re committed to ensuring this does not ever happen again.
We know that saying sorry may not be enough. We also know we work very, very hard to do great things for Boise State, and we will keep forging ahead in our efforts to advance Boise State educational and research experiences through high quality, innovative technology solutions.
We are sorry, and we’ll do better.