360

You guys are hardcore. Thanks for sticking around for our very last presentation on the CMS Tech track. My name is Brian Hirano. I'm joined by Distinguished Researcher Professor Nam Sung Kim of the University of Illinois. And we're here to talk about something a little less technical. There won't be a whole bunch of TLAs, FLAs, a lot of tech speak here, but talking more about CMS as a community and what we're trying to do in terms of interacting with academia and what we've done in the past year. And then Professor Kim will be talking more about his views, given his extensive research that he's done with industry, about what he thinks that we need to do as an industry, as well as CMS, to be able to help industry and academia succeed in what their needs are.

So, having been at this for a while in industry, you kind of know that every time there's a new technology that comes out, there's a lot of hype around it, both in industry and everyone's trying to figure out how they're going to use this thing, how they're going to shake it, what applications apply. Academia is no different. There's a couple of reasons for that; one is that often times when they're building a new product, the industry folks are more than willing to give money to academia to help answer the very same questions, but at the same time, the people in academia are trying to make sure that their students are relevant. Professors like Professor Kim are trying to make sure that their students are doing relevant work that will not only influence academic research, but also potentially influence industry and give valuable feedback to industry. And CXL is no different; CMS was created originally as the software-defined memory workstream to address some of the issues with CXL. And often times it's hard to go back and figure out the progeny of all this research. But for those of us who have been at this, the CXL thing for a while, we kind of recognize these papers as kind of the starting point for research around pooling, tiering, and how to be able to address and mitigate some of the issues around CXL.

But it's a one-way street. Most industries' view of how academic researchers work is kind of very unidirectional in terms of academic research. Researchers are - they may acquire some of the hardware for which they research. They might model some of the characteristics of devices that are new to industry. And based on that information, they generate a bunch of papers; industry views the big academic conferences as filtering for some of those papers as they go through program committees that are staffed with their peers, hopefully with a few people from industry. And at the end of it, they get a bunch of papers that then industry considers. Without realizing, many people in industry don't realize that a lot of academic researchers are willing to take feedback and, matter of fact, are hungry for feedback to know if their work is relevant or not. And it's in the spirit of this that Daniel Berger from Azure Research and I talked about forming this academic work stream within CMS to be able to help track all these papers that are going on around CMS-related activities, CXL, memory tiering, memory pooling, et cetera; well, as you know, how do we get more active in participating in the academic community to give them feedback.

So, this is just an overview of some of the things we've done over the past year. An industrious group of people within the CMS team have been reviewing papers, and based on those reviews, we invited academic researchers in to present in the CMS meetings. Sometimes it's hot off the press kind of research where it was immediately published and the next week after, or sometimes even before the research gets published, we get presentations and things like that. Just to have the industry people recognize that there is a very relevant research going on, and for the researchers to get very direct feedback from the questions from the CMS community as well as have side conversations. You know, for the past meeting and longer term, of course, we want to be a little more active in how we deal with industry or academic research. Maybe we can do other things, but we've done quite a bit of work in our CMS tech day that we had at Meta, which was hosted by Meta this past May. We actually invited six researchers to present, and four of them actually came in person. One person came from one group from Korea to present at CMS tech day. And I think one of the findings that we had was that the academic researchers were actually doing much more advanced work than some of us were doing in industry, which was a little eye-opening. So, in order to be able to leverage those things, it is very important to be able to listen to what academic researchers are talking about. So on that note, I'd like to head it over to Professor Kim to give his perspectives on how industry and academia can collaborate better.

Thank you Brian. So I spent almost half of my career in industry and in academia have, and often you know from industry, we hear that oh, what you guys are doing is useless, you know, bogus, and bullshit, and so on, and why don't you do something more useful? It's kind of ironic knowing that, you know, most driven and talented students in the graduate school program go to academia, and that they also want to impact the industry. But somehow we couldn't. From time to time we do that, but mostly we don't.

So for this, I mean, I want to advocate academia first. Of course, it's all you know; we need to blame industry, academia, and the government all together, in my view. So blaming industry side, I want to complain that, you know, industry doesn't intellectually contribute to academia, you know. It's totally understandable. I mean, you know, they have to make money and competition is steep, and so on. But, you know, many big tech companies don't publish any papers at all, you know, even after a few years. After, you know, they release the product and so on, then there's no need to keep the secret. And from our point of view, to conduct useful research, we have to know the real problem, what the real problems are. But, you know, all these industry people rarely talk about these problems; don't show the workload, they're so secretive. And, you know, thankfully, not all companies are like that. Very few companies, especially, you know, SRC member companies, yeah, they actively engage us and try to help us and so on. But majority of the companies don't do that at all. And at the same time, they're complaining that we are not producing the workforce that they need.

And I'm also complaining, I'm also blaming the industry and, you know, government altogether. So, lack of a financial contribution. Not many people know about this, okay. But the NSF budget, you know, primary source of funding for academia, has been stagnant for the past 20 years. At the same time, we have a lot of demand for student IT side, and we've recruited a lot of professors. Now, for fixed pro-money, a lot more people are competing for that. And this is from NSF data: Expected annualized award amount per proposal, which includes multiple PIs, is $50,000. The funding rate is, you know, close to 10 percent. So, even for me, submitting 10 proposals—each one takes a month. Only one gets funded.

Now, let me talk about how much it costs to support one graduate student per year. University of Illinois, it’s a public school, so our tuition rate is much lower. Private schools, even higher. But still, supporting one graduate student per year costs $80,000. $80,000, and we get only $50,000 dollars per proposal, on average. And that money has to be divided, you know, between multiple PIs. Now, I mean, what I'm going to say is, you know, mostly relevant to U.S. institutions and, you know, circuit, EDA, and the computer architecture area. So, as a result, more professors are changing. And they’re spending more time chasing the money. Now, what's happening in the academic community, you know, we have these, you know, top-tier conferences, MICRO, ISCA, S&P, we’re in crisis. We, you know, used to dominate these conferences in the U.S., but now, more than 50 percent of all papers, top-tier conference papers, are from Asian countries. We are losing the game here. It’s crisis, in my view. At the same time, our tech companies, I’m not going to name all these companies’ names. They are making billions of dollars in profit, but they don’t bother to give us a meaningful amount of money, or not at all. And they just cherry-pick the students that we, you know, trained with hard-earned money. And there are some companies that are still faithfully, you know, providing the funding for the companies, but they are not being able to recruit those students that they funded. Because those companies, you know, they don’t contribute anything. They just give a little bit more offer to those students, and they don’t have a choice. They will pick, you know, a better offer. And this will ruin the whole ecosystem, educational ecosystem, in my view.

And I'm also blaming ourselves. Often, I'm in the, you know, promotion and all these tenure committees, too. But the university also measures the performance of professors based on the number of papers. Of course, it's hard to measure the true impact in a short-term time period, but that's what it is. And while, you know, circuit, EDA, computer architecture, they are the backbone of this community. But since we don't have much money, universities are not willing to hire more professors in this area. It's just, you know, hiring some quantum people, machine learning people. We are suffering.

So, I think, we have to, of course, we have to come up with a better model, a collaboration model between the university and the industry. Traditionally, industry just gives us some money, and sees how we do. But I think it, you know, starts to change the model, and we need some mutual trust, and less stringent NDA. And one option is industry should help academics to access and explore new technology, like CXL, before the commercial release, and let them be a beta tester. I mean, we can do that with, you know, NDA, and in this country, at least the NDA is honored, and we try to keep that way. Also, academics should understand. I mean, we are very eager to publish the papers, but we also have to understand that some things are not publishable. And we have to wait until the technology becomes, you know, available to everybody. Then, you know, we publish the paper. So, I have a pretty good success story with Intel. And the Intel kindly allowed us to access the Intel CXL devices. And, you know, we had the access two years ago. And we worked together with Intel, debugging a lot of features. But Intel also helped us to, you know, develop new features. And we kept that work until it becomes commercially available. And we published them. And Intel also appreciated what we contributed, and that this, whatever the lower right side of the article, from the Intel website, recognizing what we did for them, and that they used what we developed for their, you know, fifth-generation Intel Xeon processor launch event.

I also understand why some companies are hesitant to work with the professors. One of them is, you know, one professor working with multiple companies; they are, you know, afraid of leaking, you know, information from one company to another. Okay, that's totally understandable. Then, we can do something like this. I know, you know, a few companies did this in the past. So, a company provides the following on the mutual agreement. So, basically, a company is buying out an entire lab. So, that that professor doesn't need to work with other companies. As long as we get enough funding, I don't have any reason to work with other companies either. And, I mean, a company also should provide some done university space and, you know, part-time appointment with the professors and students, so that the IP issues become less troublesome. There are precedents. And, you know, some professors, like me, you know, who want to impact the industry are willing to do this and negotiate with the university. Although, I mean, we need, I mean, for this kind of model, we need more debate and, you know, consensus and, you know, also, you know, it's not easy. But, I'm trying to bring up the problem that we are facing these days and, you know, provide some potential solutions. So that, you know, this area in the U.S. can strive and impacting the industry in the future. Thank you. That's all.

Sorry, I have to finish. But, at the same time, you know, just to, so it kind of gives you some insight into some of the topics that Professor Kim and I talk about in our phone calls. So, again, what we're trying to do in the original spirit of what Daniel Berger and I talked about was we're trying to build a community. And, we have to be conscious of the requirements of our partners, whether they be other people in industry or academic researchers, on their needs and their requirements. Because, if we both benefit, we all benefit, right? So, call to action here is, well, we have this academic research work stream. If this kind of work resonates with you, please join us because you do get to work with some very smart people. And, you had the pleasure of meeting very smart students. And, someday, those students might be your employees or they might be some professors elsewhere who produce new students. That then might become your employees. You just don't know how it works. And, the other thing is, for my industry collaborators here, you get what you put into it. If you just stand on the sidelines, you're not going to get involved and you'll have no idea of what, you know, what you may be able to get out of it. So, give it a shot. You know, we're trying to do various things, too, that are more directed, that are more sort of participatory, more active, instead of just reviewing papers. And, I have an idea for doing the, with some sort of memory touring contest. As well as trying to do things that were inspired by some of the things with the silent data corruption people. That they actually funded six proposals at various, for various researchers at, you know, they were some industry sponsorship. But, they were quite substantial in terms of the amount of money that they gave to these proposals, to fund these proposals. I talked to one of the professors because I recognized him. And, he was very happy with the number they gave because, you know, it's nearly funding one student. So, I'd like to be able to sort of participate in those kind of things, too. And, the other thing is, we're not doing this in strictly, this isn't just Brian going off and doing these kind of things. I do have a conversation with the people in the SDC work stream. I have conversations with Bijan and the foundation staff about these kind of things. And so, making sure that we're in sync here. And, it's actually a very active, even in the foundation staff, about how to interact with university researchers. And, we actually had a meeting earlier this week to discuss that, too.

So, please join us. And, I'm not the only one doing this. There's a bunch of people here that hopefully are still here that I'd like to recognize that have been actively helping me and the CMS community with the academic research. I wanted to do two things. One is to recognize them, and just to make sure that hey, we've got some work to do for the end of the year. So, please stick to this. Anyways, with that, please take questions. And, Professor Kim doesn't normally attend these CSF meetings, so please direct questions to him. Thank you.

Thank you.

Hi, my name is Paul Boreal. I'm the CEO of Daedalus Corporation. And, over my three and a half decades of career in CSF, in my senior executive positions, I have given probably several million dollars in academic funding to different organizations from the Santa Fe Institute, Caltech, MIT, Carnegie Mellon, and I think about another four or five others. So, I've been on that side of the fence. And, I'm now on this side of the fence as a CEO. And, I have to deal with problems like NDAs. And, the thing that my CFO and I have the most difficulty with is negotiating NDAs with academics. And, a lot of people don't know the reason why we do it. My philosophy as a CEO is that most of what we're going to do is going to be open source. So, then people would say, "Well, why not bother?" Well, it turns out that we live in a very hostile world when it comes to IP. So, it's not so much that we're trying to keep everything safe. It's not so much that we're trying to get into ourselves and be greedy. It's because we're trying to fend off potential attacks from people who discover that we're suddenly going to become a really important company, and they try to come after us on IP. So, I don't know where that discussion could go. But, one of the issues is, for example, if you've got a bunch of ideas that your engineers have had, and your academic partners are participating in those discussions, we have to have some kind of NDA.

Of course.

Otherwise, the USPTO does not recognize that we have a valid patent. So, this is a very challenging conversation. What's your thought on that?

So, in one of my slides, I say we need an NDA. We need an NDA for this kind of collaboration. And, there are plenty of open-minded professors who are willing to do that. And, also, it’s up to individual professors and PIA in terms of convincing the university to accept the NDA terms from the companies, including IP terms, too. I don’t care about the IP too much, and the IP becomes often an obstacle between, you know, the collaboration between the companies and the universities. But, there are a lot of people who are willing to accept such NDAs and IP terms so that we can do more useful things, more impactful things, changing the industry, and so on. So, yes, NDAs are a must. I’m totally with you.

I would love to see a good NDA that's been well thought through and negotiated and checked by lawyers and made more standard across the whole industry. That would save me a great deal of time negotiating because I'm not a lawyer; I'm an engineer. I used to build things. And so, if I could go to ChatGPT and just say, please give me, you know, a nice standard friendly NDA for an academic collaborator, that would be great. But, ChatGPT gets its own title.

I mean, I have been, you know, negotiating the NDA with many companies. And the university usually listens to me. Oh, are you willing to take this risk? And it's actually up to the professors. So, yeah.

The thing that I value the most is the collaboration. Let's make that really clear.

Yeah. One important thing here is, as I said in one of my slides, mutual trust. You can start small and, you know, you work together, you know, and see how trustworthy that person is or the counterpart is. Then you can increase the intensity and the level of the collaboration on the NDA, of course.

Thank you.

Yeah.

So, one of the things that I want to talk about is that you have to kind of start with senior researchers that can negotiate with the university on your behalf for that kind of thing. So, that's the other aspect: it's one of those things that there's a big problem, and how do we chip away at it, right?

And another issue with the younger generation of professors is that they need to get promoted. They are very desperate for writing papers, but, you know, people like me, I don't need more papers. I want to impact the world. Then we are more willing to accept such terms, and I don't care whether I can publish something as long as I can do something useful. There are a lot of people like me, yeah.

A very, very interesting conversation, and I appreciate all the academic work. A lot of interns I worked with, they always bring a different angle, and they are very motivated. So, one suggestion I have based on what I heard is: we have a focus group here, OCPCMS, and we are all focused on certain problems to solve. And we all post interns to our respective companies. So, maybe through this channel, you can funnel all internships through this channel. Then there is one dedicated group of interns in various companies, and that may help.

Yeah, it is great, but also it is kind of a double-edged sword in my view, unless we have a proper relationship between the professor who is sending the intern to the company and the company. Because they have to make a consistent progress under some consistent theme to graduate. Right? Then I think the most ideal case is either: the student brings his idea to the company and continues to work on it, or the company gives the idea or, you know, some initial work. But the company should allow the student to continue to work on it to publish whatever ideas or, you know, getting his thesis done.

Got it.

Yeah.

Okay.

So, I probably say it is a long-term aspiration because if you look at most of the places that do the internship kind of things, they are under existing umbrellas like SRC Jump or industry-sponsored consortium like CMU Parallel Data Lab or something like that, where industry is investing in it already. And so, as a benefit of that, you get access to the intern. So, if we can get to the point within OCP that we have a little more umbrella around the research thing, and then we can somehow figure out what that participation looks like, then we can do that. But certainly, if a professor sends me an email and says, "Well, I am looking for internships," I am more than happy to facilitate. So, even though we might not be able to do it organizationally, certainly I would do it personally just because I have the interaction. And which is, you know, a potential benefit also. So, not as formal, but there is a benefit.

Any other question?

Tushar Krishna from Georgia Tech. So, no question, I just wanted to kind of first thank you both for bringing this up. And I think especially the point Nam Sung brought up, I echo the fact that I think there is a disproportionate thing here where there are certain companies that, again, traditionally have been funding research. They are doing it. Most of our students don't end up joining those companies.

Right.

They did like their entire PhD, but, you know, they ended up joining some other company because they were paying more. So, it almost feels like, you know, there should be some parity. Because my worry is, when this company at some point will stop funding because, you know, ultimately they have to meet their business needs and then we will really be...

So, that's because we don't do our portion of the job. So, I can give you one example. One of my, you know, top students got funded by Intel's money. So, by the time that he graduated, he got the offer from Intel, but also Meta, Google, all these, you know, companies. And I had this conversation with the Intel hiring manager. "I really appreciate what you have done for us, but I cannot make him sacrifice his financial gain. So, how about you match the salary that either Google or Meta is offering? Then I can make sure that he will take the Intel job." And I, you know, go to my student. "Same. I'm not going to force you to take the low offer, but as long as Intel can, you know, make the same offer. Will you, you know, be willing to join Intel?" Because you have worked with these people for your entire PhD. You love them. You learned a lot. It's time for you to pay back.

That makes sense.

Yeah. So, that, you know, one of the reasons we don't do our job properly. He went to Intel. Yeah. He took the job. Intel really appreciated that. So, we need to be more proactive and also advocating for the companies supporting us.

And therefore, we need to be more proactive as industry.

I think we are after time, but just a few thoughts. So, I'm really glad about this story. This is, you know, this made my day in addition to everything that you, because indeed, we, and I think this is maybe a message that the hardware company should hear: that the salaries that are being offered by the hardware companies do not match the salaries that are coming from the software. And this is a, I mean, it's a problem, you know, all the way in the pipeline, you know, attracting students to work in hardware. You know, my VLSI design class, you know, 10, 15 years ago had, you know, almost 100 students. Now, it's down to 20, 30. So, it's a problem. So, about the internships, I think that's also a double-edged sword because many of the students that go on internships are asked by the companies to sign NDAs. Even with me, they cannot share.

No, they don't.

Yeah, so that becomes actually, you know, a problem if there is not, you know, a wider, broader arrangement.

Yeah, so we have to, professors have to sign the NDA together.

Right, right, right.

And more actively engage with what student does there so that he can be doing useful thing for both the company and his own, you know, graduate school research.

Right, right. Yeah, two ways.

Yeah, yeah.

Because, yeah, I'm also very, I mean, you know, I'm here, so.

Yeah.

But, yeah, many companies would rather just have the student for, you know, for the internship.

Yeah. All right. So, that's the bitter end. I appreciate your time and thank you for letting us present. All right.

Thank you. Good job.