3D InCites Podcast

IMAPS Symposium 2025: Chiplets vs. Dielets and the Truth about Co-Packaged Optics

• Francoise von Trapp/Multiple Guests • Season 5 • Episode 19

Share episode

Send us a text

We sit down with Dr. Subu Iyer of UCLA to unpack chiplets vs dielets, why a universal ecosystem is missing, and how sub‑10 µm bump pitch could make protocols optional. Then we host a panel featuring John Knickerbocker, IBM; Mike Kelly, Amkor; and Tolga Tekin, Fraunhofer IZM on co‑packaged optics, bandwidth, and power for AI data centers.

• chiplet as design construct, dielet as physical die
• lack of universal chiplet ecosystem and interoperability
• bump pitch scaling and protocol overhead trade‑offs
• packaging purpose reframed as power, communication, and cooling
• economic shift and value capture in advanced packaging
• national competitiveness, prototyping access, and talent pipeline
• co‑packaged optics definition, drivers, and cost targets
• copper reach limits, latency, and bandwidth density for AI
• hyperscalers as early adopters and five‑year outlook

Learn more at imaps.org


IMAPS International
IMAPS is the largest society dedicated to microelectronics and electronics packaging advancement.

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Support the show

Become a sustaining member!

Like what you hear? Follow us on LinkedIn and Twitter

Interested in reaching a qualified audience of microelectronics industry decision-makers? Invest in host-read advertisements, and promote your company in upcoming episodes. Contact Françoise von Trapp to learn more.

Interested in becoming a sponsor of the 3D InCites Podcast? Check out our 2024 Media Kit. Learn more about the 3D InCites Community and how you can become more involved.

Francoise von Trapp:

This episode of the 3D InCites podcast is sponsored by IMAPs, the premier global association for microelectronics advanced packaging enthusiasts. A membership in IMAPs helps your company grow its advanced packaging workforce through professional education and networking, advances your brand, and supports building relationships. IMAPS helps you learn, connect, and collaborate. Learn more at imaps.org. Hi there. I'm Francoise von Trapp, and this is the 3D InCites Podcast.

Francoise von Trapp:

Hi everyone. This week we're recording live from the IMAPS International Symposium in San Diego, and we're talking about, as always, advancements in the advanced packaging space or electronic packaging or microelectronics. And today, Subu Iyer, Dr. of UCLA gave a keynote talk titled Strategic Directions for Electronics Packaging. And Dr. Iyer was formerly with IBM. He's also been with the NAPMP. He's been back and forth to UCLA. And he is the very first person I ever heard present on the concept of disaggregating system on chip technology into its functional parts and re-aggregating them, which thus became what he calls dielets, which everybody else calls chiplets. And I noticed from this pre presentation that you're now calling them chiplets.

Subu Iyer:

Or dielet.

Francoise von Trapp:

Or dielets. So welcome to the podcast, Subu.

Subu Iyer:

It's great, uh Francoise. I'm uh so happy to be here. I I've been giving this podcast uh a few times, and uh and I also noticed that it is very popular, and I listen to many of your podcasts as well.

Francoise von Trapp:

Well, I appreciate that. I'm so glad. You know, I've been part of this industry for 20 years, and I think I first heard you probably when I started 3D Insights in 2009. You know, I've always loved your talks because you are a professor and you know how to hold a crowd, and you were actually saying today that someone told you don't take your knowledge giving seriously, that you really have to be an entertainer.

Subu Iyer:

That is correct.

Francoise von Trapp:

So and I've met your students. Remember the time we had dinner?

Subu Iyer:

Yeah, yeah.

Francoise von Trapp:

And I met all your students. Absolutely. You won a Research Institute of the Year award one year for your work in UCLA TV.

Subu Iyer:

From from CD3 Insights, yeah.

Francoise von Trapp:

Yeah.

Subu Iyer:

So And that was uh right after we uh very very soon after we started our within about three or four years of our uh I came to UCLA. So that was actually a big deal and meant a lot to us and to our students. Thank you so much.

Francoise von Trapp:

You're welcome. I loved your talk this morning. I want to hit on some of the highlights. Subu went through pretty much the history of advanced packaging, and we're not we don't have time to do that here. But one of the things you said is there are no chiplets. So what did you mean by that?

Subu Iyer:

Yeah, I think the concept of chiplets, you talked about like pioneering this whole thought process of um disaggregation, right? And the key element of the disaggregation is a chiplet or a dialet. And actually the two are mean slightly different things, okay? A chiplet is actually a design construct. A dialet is a hard piece of silicon, all right? And so when we integrate it on a substrate, it's a dilet. When we design it, okay, it's a chiplet.

Francoise von Trapp:

Okay, that's the first time I've heard that.

Subu Iyer:

Yeah, so it's it's the same difference between a chip and a die, right?

Francoise von Trapp:

Right.

Subu Iyer:

A designer designs a chip.

Francoise von Trapp:

Right.

Subu Iyer:

He sends it off the data to the foundry, the foundry then makes a wafer, the wafers then cut up into dyes. Right? We don't cut up the into chips, we cut up into dies. The dyes are assembled, and then they again mysteriously become a chip.

Francoise von Trapp:

Oh my god. You just blew my mind. That is again the first this is what I love about you. You know, that you you explain things so well for people who are not really technical. And I s imagine this is because this is how you address your freshman class.

Subu Iyer:

And my management.

Francoise von Trapp:

And your management But it's good because somebody needs to be able to do that. You know, there's no clarity that we're looking for it.

Subu Iyer:

Oh, let me go back to this question uh which you asked, which we kind of got sidetracked on, which is I said there were no chiplets.

Francoise von Trapp:

Right, okay.

Subu Iyer:

Now that has to be taken in context, okay? There are chiplets. There is no triplet ecosystem that is universal. Okay. In other words, let's say, you know, company A produces chiplets that they use in their own systems. All right. Company B produces chiplets that they use in their own systems. And the company B triplets don't work with company A and vice versa. Right? So when I say there is no chiplet ecosystem, it what it means is that if we had a student, for example, who gets up one morning with a brilliant idea and says, Hey, I want to use this company A chiplet and I want to integrate it with company B triplet and get some other chiplets from somewhere else and put it all together and build this really unique system, he cannot do that. Okay, he cannot do that because these two chiplets from these two different companies and any other company do not talk to each other. And uh they have different mechanical standards, they're different sizes, and they don't like fit together nicely. All right.

Francoise von Trapp:

Because they weren't designed to fit.

Subu Iyer:

They were not designed, they were designed to work in their own ecosystem.

Francoise von Trapp:

So isn't that the difference between a system and a package where you're taking different functionality chips and just interconnecting them closer together? And because you said before the chiplet is a function of design. Right. Correct. One of the things the industry has been working on is that interface, right?

Subu Iyer:

The UCIER Yeah, there have been many interfaces that have come and gone.

Francoise von Trapp:

But okay, so what you're working on at UCLA then, too, are you promoting this ecosystem?

Subu Iyer:

We are a university, right? And so our goal is not some proprietary system, right? We want to kind of dispense knowledge, we want everybody to be able to work with each other, and most importantly, we want to encourage rather than stifle innovation. So today, right, if you look at this, right, the number of people who are building integrated systems, right, from scratch are very limited. Okay, and that's because, you know, first of all, the chiplets are not available. Secondly, nobody's going to talk to you if uh unless you are willing to like to make a lot of parts, right? I mean, if you're thinking about something like doing a prototype or building a small number of parts and so on, customizing for like a specific application, you're out of luck. Okay, nobody will even talk to you. However, okay, if you have a platform that is universal, think about Lego blocks, right? So, Lego blocks, you have different kinds of Lego blocks, but they all have one characteristic they fit on each other, right? Right? They may have different colors and different sizes and whatnot, but they all fit together.

Francoise von Trapp:

They also have some different shapes.

Subu Iyer:

They have different shapes and so on and so forth, right? Now, we probably don't want to have chips of different shapes, okay, but you know, the point here is if you have like uh standardized sort of dimensions, right? X dimensions and y dimensions, and you have standardized mechanical characteristics, standardized bump locations, and so on and so forth, then every chip can talk to every chip, okay, and uh standardized protocol, right? And the protocol and what happens, and the thing that I was trying to emphasize today was we can actually discard protocols once we make the dimensions of the bump pitches really, really small, and really as small as like sub 10 microns, which is possible today. So without sort of if you just let this thing flow like it has been flowing, we will within the next one or two years reach a point where say people say, hey, we don't need UCIE, we don't need this, we don't need that. We can just use the native interface between them between these chips and continue as if it was the same chip.

Francoise von Trapp:

Okay.

Subu Iyer:

Right? Because now there is no difference between communicating on one chip and communicating between chips.

Francoise von Trapp:

So is that what you mean by packaging being an extension of monolithic chips?

Subu Iyer:

Exactly. So like when you design a monolithic chip, right, we have these IP blocks. And these IP blocks are designed by various people, okay, and they're available as code, right? Basically. They have things like descriptors, like a VHDL descriptor and a.lef descriptor, which is uh talks about the physical locations and so on of where the pins are for that particular IP block, and then you synthesize a chip with that, with those IP blocks, right? And you're downloading you're downloading code to actually integrate this thing. So now when we transition to like the chiplet or dialet regime, you're no longer dialet downloading code, you're actually downloading real hard dies. But if these dies can be described the same way, like with these abstractions, they call abstractions, and these abstractions are like functional descriptions of the chips saying, hey, look, pin number one does this, pin number two does that, and so on. And these pins can communicate with pin number A on the other side, pin number B on the other side, and basically all that you need is to connect them with a wire and life goes on as normal, then you have achieved an open ecosystem for these chips.

Francoise von Trapp:

So your prediction is that we won't need the UCIE at some point.

Subu Iyer:

Correct. We won't need the UCIE. I mean you can still use UCIE and whatever things you want, but they're gonna come with their own overheads and you can achieve the same result even without using it.

Francoise von Trapp:

Right. Okay.

Subu Iyer:

But if you have something that already has UCIE, you can still use it.

Francoise von Trapp:

Okay. One of the other things you said that stuck with me, um, not on the chiplet topic, but maybe on the chiplet topic actually, is that the change in emphasis for what packaging is has gone from being the protection of the device to power.

Subu Iyer:

Yeah, power delivery. Communication and cooling, right.

Francoise von Trapp:

And this is all to address this terrible power-hungry data center activity. I love that you use the example of doing a query for what Taylor Swift is wearing today and generating how much power that causes. People are out there, the general public, don't understand these frivolous uses and how much energy it takes.

Subu Iyer:

So I think, you know, one has to be careful, right? I mean, think about how we did this in the old days. Like I would have to walk to the library or take a bus to the library or w or some some sort of mode of transportation, then I would go check in, then I would have to go search for the books, then I would have to open the books and find the page and and all that stuff, right? Now that too takes energy.

Francoise von Trapp:

Yeah.

Subu Iyer:

All right. Yeah. And the amount of energy I need to do that, right, just driving to the library, right? Right. Is a lot more than this uh energy, right? So you have to look at it from that perspective. Right. Okay, but I think what happens now is because this is so easy, okay, we use it in trivial ways. Like, you know, how does it matter what she's wearing? Okay. Right, right.

Francoise von Trapp:

Well, that's what I mean. I mean, it's one thing to use it for research.

Subu Iyer:

Yeah, I think, right? Uh, overall, we're all very curious people in general, right? We will continue to use this. If you can do it, you will do it. Right. Right? So the question now is not how do I restrict the use and say, if somebody asks uh what is stalysting, I'm not gonna answer. Or Gemini is not gonna answer. That is not the right answer. So the right answer is to say, how do I make the energy required to answer that query as low as possible? And that is where I think advanced packaging plays a big role.

Francoise von Trapp:

Well, and that's what our industry can do, right? It can solve it from the perspective of let's just make it require less energy versus restrict people from using it frivolously and can contributing to global warming, right? Exactly. Do you think that's one of the reasons that there is a bigger focus by the front-end guys are starting to realize that advanced packaging is about more than just protection of chips?

Subu Iyer:

Yeah, I think so. I think there's a better and better understanding and I of what the value that packaging brings, right? The value that packaging brings is far greater than just like uh packaging a chip like in the good old days. Now it is actually delivering a system. Right. Right? And so the value added packaging now is significantly in a different dimension than it used to be. Right. And there, and that's the reason I said, right, I mean, and a few people came to me after the talk. I love it. We you know we're sick and tired of this 1.5%, 3.5% gross margin. We own also 50%.

Francoise von Trapp:

Right.

Subu Iyer:

And TSMC figured that out and said, okay, you know what? We're gonna actually have uh this advanced packaging, it's gonna be like out of this world, and we're gonna get 50% gross margin. So they are the TSMC is not doing this in because of philanthropy.

Francoise von Trapp:

Right.

Subu Iyer:

Okay.

Francoise von Trapp:

I mean No, they're a business, they want to make money, I get it.

Subu Iyer:

And I think, you know, I think what the packaging guys should do is they have to get out of this mindset that I have to make everything cheap, you know, I can't use advanced uh technology because that's expensive. You have to ask yourself, what is the value that I'm bringing? Right. And if that value can justify the cost, which clearly does, right, uh, I think there's a market.

Francoise von Trapp:

Right. Yeah, I completely agree. I've been trying to say that for a long time. You know, I'm not arrogant enough to think that my belief in this and my evangelizing for years is that I'm not sure. I don't have that problem. You know, for the well, no, but you are scientist, you're you are you are an engineer, you understand this fundamentally, and you have made this happen over the years. I've only been the voice of all of these thoughts, and I believe it, and I'm really excited to have contributed to that just by getting the message out there because I think that you know, messaging it takes a while. But, you know, I saw this, I understood this a long time ago, but realized that the the industry wasn't catching on in the sense that it takes so long in this industry for people to realize something that to me seemed really um, you know, you can't see the forest for the trees kind of thing.

Subu Iyer:

When you're in the weeds, you can't see anything.

Francoise von Trapp:

Right, and so it was kind of my job to say it over and over and over again.

Subu Iyer:

No, absolutely. And I think, you know, I'll I'll say this like um uh I remember when I was IBM and I talked to some of my colleagues, very experienced people, very, very respected people, and I said, hey, look, you know, how what do we need to do to make the bump pitch smaller? And uh a very, very uh respected guy comes and tells me, Hey, and he was an IBM fellow and all that stuff, okay? So very, very well known guy. He comes to me and says, You know, Subhu, you're full of, you know, this. First of all, it's very difficult to do, but even if you could do it, nobody needs it. We've been doing it this way for the last 40 years, and there is no need to change.

Francoise von Trapp:

And that is the mantra of this industry. We change, we don't change anything until there's a need to change it, until it doesn't work anymore, and then we have to develop something new.

Subu Iyer:

Yeah, and another sort of thing that came about along the same lines is like roadmaps. So when you're in the silicon business, I come from the silicon end of things, right? So I I went and I suddenly became the uh director of packaging, and that's a funny story itself. Okay, uh, I actually was I had a small group and was working with a company called Micron, uh, which was um uh sort of building this 3D stack memory, and they came to us to work with us, and so you know, I was trying to get all this done done, TSVs and all this good stuff, right? And I was having such a hard time with the packaging guys, the classical packaging guys. They said, Who wants this? This is never gonna work, blah, blah, blah, and nobody needs it. And so I would go to my boss, who is a guy called Gary Patton, and I would say, Gary, you know, somehow a lot of problems. One fine day, the guy, you know, Gary just blew up and he said, You know, so I'm tired of you complaining about these packaging guys. So from tomorrow, you're gonna manage them.

Francoise von Trapp:

There you go.

Subu Iyer:

Yeah, so so all of a sudden, right, I have this 300-plus uh group of packaging engineers, and now you know they kind of have to do what I tell them to do, right? And so now it is a question of getting education across and getting them to understand what we're trying to do and so on and so forth. And I think one of the things I asked them when I started this job was, hey, uh can you show me your packaging roadmap? So what they said was very, very enlightening, and I think this is a very important thing to remember. And they said, Subu, we don't work with roadmaps. Okay, because when you're in silicon, you know the roadmap is Moore's law, right? Every year we know what to do. So, but they said we don't do this. The chip guys come to us with a problem and we solve it. And so I said, and the next next guy, the next chip comes, we solve that too. And is that the same solution? No. So I you know, and so I said, look, the main thing we have to do here is we have to have a structured roadmap so that we can tell the chip guys what is gonna come about, okay? And they can plan for it, and you can plan for it, and this makes everything smoother. So I think you know that is still not happened in the even with the heterogeneous. Yeah, even with the heterogeneous T you know, TSMC does it one way, these uh the guys do it this way, and you'll see that like the OSATs, for example, are trying to get into the business, but everybody has a different um different roadmap. Yeah, okay, and these roadmaps don't talk to each other, they don't flow from one to the other, and so on and so forth. Now, and the reason why TSMC is kind of successful is their roadmap is self-consistent within their ecosystem.

Francoise von Trapp:

Right.

Subu Iyer:

And that is that works for sure, as long as you're willing to go there today.

Francoise von Trapp:

But it's also a problem because you can't have one company dominate dominate because if something happens to that company, we're screwed.

Subu Iyer:

Yeah, and it's also it's also not good for innovation, right? I mean, yeah. So I think roadmapping is good, and that's one of the reasons, right? We started this uh uh effort which is funded by the government actually called MRHIEP, the manufacturing roadmap for heterogeneous integration and electronics packaging. And that roadmap is actually on our website. Anybody can go and look at it. And we uh when when I was in the NAPMP, that was the roadmap we followed.

Francoise von Trapp:

Okay.

Subu Iyer:

Okay, and uh that is uh the roadmap I think the industry should follow.

Francoise von Trapp:

Did you ever see Back to the Future? Yeah, long ago. Yeah. Do you remember Doc says where we're going, we don't need roads. Yes. Okay, so since you brought up the NAPMP.

Subu Iyer:

Oh, okay, I and we don't have too much time left.

Francoise von Trapp:

I did want to get there, and you brought us there, so I'm happy about that. You said today you showed a slide that talked about the NAPMP structure and the six hardware eco and ecosystem thrusts and piloting facility and prototyping challenges, and then you had all these question marks across it because all of this because of the current administration has scrapped the NAPMP, but you told people don't toss this work, it's still important. Can you talk about that a little bit?

Subu Iyer:

Correct. First of all, let's be clear. I don't think the government has tossed NAP and P. They are redefining it and probably rebranding it. And every new administration wants to do that, right? I mean, uh they want to kind of make sure that it reflects their sort of ideas of what is uh what is important and so on. And and that's perfectly okay. I don't have any issue with that. I I think the new uh BAA that I have seen is broad enough and includes uh in a very broad sense, you know, heterogeneous integration, packaging, assembly tests, and all this stuff, you know, which are which are the what these these five areas used to address. So I think, you know, my feeling, okay, and I mean time only will tell, is that the ideas of the original thing are still sort of valid and fit nicely actually into the new structure. And if you look at where they're pushing, right, the the the new BAA is pushing AI. Okay, it's pushing a whole bunch of other things. But the AI push, okay, is clearly something that will be driven by advanced packaging. And you're not gonna get those results, okay, if you want to reduce the power and all that good stuff. You're not gonna get that without, you know, uh basically following the kind of structure and the ideas that I presented that I developed when I was in the NAP and P and also what I sort of briefly presented here today. You have to address the substrate issue, you have to address the equipment new equipment that is needed for this, you have to develop processes for it, you have to address thermal and power delivery, you have to address connectors, and you have to address the chip right problem, and you have to address the EDA problem. Okay, those key elements are key are still very important. And you also need, I mean, it's not enough to actually just build these in abstract. Okay, you need a place to build these. Okay, and if you don't have a place to build this, you'll have to go to China, you'll have to go to Taiwan, you'll have to go somewhere else to do it. And so it is really, really, really, really important that we provide a safe place where uh innovators in this country, and we have the best innovators in this country, can actually sort of try out their ideas with advanced packaging and be able to get parts back, test them, make sure they work, and that will lead to bigger and bigger volumes and manufacturing in the US. So I firmly believe, okay, that the strategy that we put together with my colleagues when I was in the NAPMP, and what I kind of re-enunciated today, okay, are vital for making the country competitive, okay, in in advanced packaging. And one thing is very clear nobody, no country, no uh country has a monopoly on innovation. Right. Okay. And today, right, I would say half my students come from outside this country, right? And uh it's more and more difficult nowadays with all these restrictions for these for us to accept students from certain countries. And this is going to be a very serious problem. Big, big one the way we've been innovative is we've been able to attract the best talent.

Francoise von Trapp:

Are the H1B visas impacting students? Not really.

Subu Iyer:

They don't they don't impact students, okay, but they do impact the ability of these students to get jobs. And if they don't get jobs and they don't get these visas, okay, they're going to go back to their home country. So here we are, we put a lot of effort educating them, they become fantastic engineers, and we say go back to your country. Right? And um and there are jobs in those countries, okay?

Francoise von Trapp:

And isn't it getting to the point where there's other places they can get their education besides the US as well?

Subu Iyer:

That is also a big, big danger. One of the big big uh sort of exports we have is actually higher education. Okay. We have easily one of the best higher education uh infrastructures in the world. And uh if we kind of dismantle it, right, and we are dismantling it in many, many ways, I'm not saying that the universities are not to blame. The universities have actually brought a lot of these problems on themselves by sort of not focusing on the real problems. So, for example, right, and and part of that is actually the engineering faculty who have not participated, and the science faculty and the medical faculty who have not participated in the administration of the university. So if you look at the administration of the university, it is dominated by the humanities, the philosophers, the sociologists, and so on and so forth. And they have a different agenda, okay, which is a very idealized agenda, which is very difficult to actually implement, right? Like if you think about like free buses and free food and free groceries for all. I mean, that's not happening. You cannot sustain that. Right. But the people who have to sort of uh put uh the voice of reason here are not participating. And so the universities have sort of swung that pendulum to the other side, but the reaction is also not correct because by withholding funding to universities, those people don't get government funding. Okay, they don't have research projects, their overhead is very low. So they have plenty of time to talk about these problems. The guys who get affected are people like me, okay, who basically say cannot fund research. It costs me $100,000 to fund a PhD student per year. Wow. Yeah. Yeah. So and you know, and for an experimental number uh PhD, right, which is what we need to get people skilled in manufacturing, that number is almost twice. So I computed that it takes about a million bucks to get a PhD out of the door. Wow. No, we spend a million bucks and we say go back. We're not gonna give you a job. This is complete nonsense. Okay, and unfortunately, right, the people who are in power here do not seem to appreciate that. You cannot get a guy from Walmart to be like a top-notch Sundar Pichai at Google. All right?

Francoise von Trapp:

Yes.

Subu Iyer:

I mean, and no arguments from me. Yeah, and this is the problem. Yeah, we're we're educating these people, we need to benefit. Right. Okay. I'll stop.

Francoise von Trapp:

No, I mean, I would talk to you all day about this, but I can't. So um I am just so happy to have had this opportunity to talk to you one more time. I've really enjoyed our conversations over the years, so thank you for joining me today.

Subu Iyer:

Yeah, and it's been a pleasure talking with you and uh having this free free-rolling conversation. Okay.

Francoise von Trapp:

And then people get to listen to it. So that's the fun. I mean part. All right. Okay, thank you so much. Thanks, Subu.

Subu Iyer:

Bye.

Francoise von Trapp:

For several years at the IMAP Symposium, we've been following the evolution of co-packaged optics as they make their way to commercialization. Now, this year, Jan Varneman focused her Wednesday evening panel on the topic. The topic was building the ecosystem for co-packaged optics. And joining me to talk about this are three of the panelists, and they're all from our member companies. I have Mike Kelly of Amcor, Tol Liteken of Fraunhofer IZM, and John Knickerbacher of IBM. Welcome to the podcast, guys.

John Knickerbocker:

Thank you. Thank you.

Francoise von Trapp:

Before we dive into the topic of the day, can you each just tell us a little bit about what you do at your companies and how you've become involved in co-packaged optics, why you're interested in it?

Mike Kelly:

Uh hi, my name is Mike Kelly. The question is how do I got involved in Silicon Photonics? So I think like all of the projects that we undertake, it's something that our customers are interested in, and we need to get prepared. So uh Silicon Photonics, as everyone knows, has been around for a long, long time. We've worked on it on and off for you know years, I would say around a decade probably. What's completely aggravated the situation is all this bandwidth requirements in the data center for AI, and we're looking to help our customers handle that in uh in a better way than what what is possible today, staying completely electrical. So I think that's kind of the background on it, and uh, you know, we're looking forward to making progress.

Francoise von Trapp:

Okay. Tolga?

Togla Tekin:

Hi, my name is Tolga Tolga Tekken. I'm in charge of the photonic plasmonic systems group in front of RyCM. Background optical signal processing. I get introduced to photonics in back to 95 since then working on photonics, plus the on packaging side. So what we observe is actually the main bottleneck, off-chip, off-core interconnects having high bandwidth, low latency, and high density interconnects. So the promise is that photonics helping out with the entire development coming from telecom, right?

John Knickerbocker:

That's right to leverage that.

Francoise von Trapp:

Okay, and John?

John Knickerbocker:

So uh maybe I'm a little different. Uh my name is John Neckerbacher, and uh um I'm with IBM Corporation, and I've been uh developing advanced packaging technologies for over four decades. And uh and so uh a part of that, uh on and off through the career has been uh various um uh test vehicles that that uh drove uh photonics and and uh optical interconnect. And over the last uh several years, uh I've I've led a team that has uh Been driving some of our um next generation technologies, as we call it, uh, in terms of CPO technology and advanced photonics. And so, you know, I don't actually consider myself a photonics expert. I'm not but I'm a packaging expert, and we have many uh photonics experts uh uh on our team. And and so, you know, the complexity of photonics and and co-packaged optics is such that you really need to have a broad range of experts. This technology is uh has caught fire very recently, but we started this more than a decade ago, back in the 2009 time period, and uh these technologies kind of evolve from a laboratory level, and uh ultimately we want to go from laboratory to prototyping and manufacturing to help ourselves and clients meet their needs. And so that's how I've been involved, uh helping to drive this over the last several years.

Francoise von Trapp:

You know, as long as I've been in this industry, which is 15 to 20 years now, um when I started hearing about photonics, from what I understand, the issue has always been on the packaging side. You could do chip to chip. The challenge was getting the signal off the package. You know, the conversation over the years has ebbed and flowed as developments happened and also you know, drivers came along, and it seems like now we've got this, as it always is in the advanced packaging side of things or in the industry when there's a need for it, that suddenly there's this burst of activity because now there's no other way to do it, or it's going to solve a big industry challenge. So, just for the purposes of our audience who may not understand what co-packaged optics are, we've got the silicon photonics. Um, what do we mean by co-packaged optics?

John Knickerbocker:

I've heard many different versions of what people think co-packaged optics is, and I I um uh looked and found an ANSIS blog from uh 2024, and I thought uh it very well represented what uh co-packaged optics is, at least from my vantage point. This blog basically calls out that CPO is uh an approach to address the growing challenges around bandwidth density, communication latency, copper reach, and power efficiency uh to support uh the next generation optics and electronics requirements. To me, that really addresses the key points for CPO.

Francoise von Trapp:

So basically, we're replacing the copper with fiber optics in some way.

John Knickerbocker:

The reach of copper is challenging to go beyond the one to two liter range, and optics is has that uh opportunity to go that longer distance and start to bring in very high bandwidth and as energy comes down to support that in a power efficient way.

Mike Kelly:

I like your definition a lot. I think that you know, when coming at it from a strictly packaging standpoint, when we're talking about co-packaged optics, it really gets down to you know the provisions that need to be inside the package somewhere to interface to the the optical path. And that's taken many forms over the last decade, but but it seems like we've consolidated on something now that makes sense uh and is extensible. And so now I think the the key really is making sure that we have a a path to higher volume so that it can be more or less mainstream. Seems like the the vision for that is now nearer in than it was ten years ago for sure. So that's kind of what it means to me.

Francoise von Trapp:

Okay.

Togla Tekin:

Indeed, the losses in ref transmission line increase with the distance, right? And for the time being, the electro optical conversion takes place at the edge of the panel. So that's in the range of twelve inches. Right? Would somehow corresponding to 10, 12, 12 dB loss over there? Around 60 gigabolt per second. That that that's the metric, right? So if it can come closer with electro-optical conversion, closer to the ASIC, which requires a higher bandwidth, so then we'll that transmission line will not consume so much power. Right? It will be saving on that end. That's that probably will yield four times minimum power saving.

Francoise von Trapp:

So the driver for going to co-packaged optics is primarily power?

Togla Tekin:

Basically the limitation of RF micro transmission lanes. Okay.

Francoise von Trapp:

There's a difference, right, between the silicon photonics and the co-packaged optics.

Mike Kelly:

Power is definitely a part of it, but what the industry needs is a lot more bandwidth for all these disaggregated systems in particular that are AI-centric.

Francoise von Trapp:

Right, okay.

Mike Kelly:

You've got to have more bandwidth for the next generations, and this is the the best way to achieve that. Okay.

Togla Tekin:

I mean, let's refer to data center switch ASICs, 50 tera, 100 tera switches, right? So the Redix on is limited. It is SARDAS. So that means for the timing, SARS speed is following the bandwidth requirement of data center switches, right? But each increase of SARDIS speed causes additional power. So and then bringing all those together requires a closer integration of a OA conversion closer to the switch ASIC minimum.

Francoise von Trapp:

What's missing from the infrastructure today that would make it possible or that that's keeping co-packaged optics from reaching commercialization?

John Knickerbocker:

So I I think there's many obstacles to really climb that curve and go from small volume research demonstrators to volume that is used broadly across many applications. So I think one of the key drivers now is AI, right? And AI training and AI inference are very important. And the bandwidth that's needed to support that effective compute and especially the energy being consumed to do even just one training um algorithm can take months at the present time and significant energy to bring out these billion parameter models algorithms that support that information that people want to access. And the problem with that is that's just the first turn on a training module, and yet you often need to go through more iterations of fine tuning. Uh likewise for inference uh operation. You also want to go through some iterations of fine tuning and be efficient in this. So as was stated earlier, I I think the value of CPO is both in the bandwidth it can bring to these applications that can reduce the amount of time and energy required for both training and inference and fine-tuning, as well as the ability to support the interconnectivity of these uh disaggregated systems across the data center at longer distances than can be done today with many, many more GPUs and modules that need to be connected in order to be efficient.

Francoise von Trapp:

So, as far as a data center application, for instance, a large language model.

John Knickerbocker:

It's more about being um efficient with energy and uh driving these very large language model algorithms are just power hungry and need lots and lots of GPUs from you know from hundreds to thousands to tens of thousands. And when you get at that scale, the uh the amount of energy that's being consumed to create these models and run them through to be effective is just too large, right? And so you can either reduce the algorithm size, right? I think there's room for that as well, or you can also uh create this higher bandwidth connectivity between these GPUs so they can be uh more effective and get to the answer quick more quickly. And each is probably the right way to go. Uh some some combination.

Mike Kelly:

I think just a little fine-tune on that is like you mentioned earlier, the algorithms are so large. Now they're trillions of parameters, and so there's an immensely greater amount of data movement in the data center, back and forth from the different memory domains surrounding all these GPUs. So the amount of data that needs to go back and forth as you train a model, or even when you infer from an existing model, is orders of magnitude larger now that LLMs have come onto the scene. This is still kind of attacking the question why all the interest in silicon photonics now? Because the amount of data that needs to get moved around between different GPUs and especially between different memory blocks in that GPU space is becoming and will become larger than you can really handle in copper, than you can handle electrically. It just we won't be able to grow the capability of data centers if we stay electrically connected. It's going to need to move beyond that to get the amount of bandwidth again to be able to tackle these larger models.

Francoise von Trapp:

Okay, so we're understanding why the why and what the driver is. So, what's missing from the infrastructure preventing us from being able to adopt this?

Mike Kelly:

The thing that's been missing for a long time is a really cost economic way to build silicon photonics transceivers so that you can use them ubiquitously everywhere. That's work in progress still, it's not a done deal. But a lot of the big um hardware companies that are working on making this possible have put the stake in the ground. I have to have this. And so, you know, I think it's energized the the industrial community. Come around and say, we gotta make this happen now. It's it's urgent, it's not optional. And to me, that's the difference today than five years ago or ten years ago.

Togla Tekin:

But so I would like to somehow differentiate, right, the traditional data center are hard hyperscalers.

Francoise von Trapp:

Uh-huh.

Togla Tekin:

The CPO concept started over there. With AI data center, sure, we need high bandwidth inside the data center, but additionally the lowest latency in order to have computing run parallel. So I can probably give some numbers on traditional data centers, hyperscalers requiring the acceptance of electro-optical transceivers from globals is expected one dollar per gigabit per second. So if you can meet that target, you are in. Otherwise you cannot be in the game. So possibly it will be similar in AI data centers, I could imagine. But since the requirements are different, that several GPUs or XPUs to be connected at the same time. It's more beyond than beyond than the co-packaged optics. So it's possibly in chiplet level to be realized. I mean, that's slightly different. I I will see more advanced thinker.

Francoise von Trapp:

Okay. So we're gonna see first implementation of co-packaged optics then in AI data centers.

Togla Tekin:

In traditional ones.

Francoise von Trapp:

You'll see it in hyperscalers first. And when do we think when do we think that's gonna happen?

Togla Tekin:

I mean if you meet one dollar per gigabit per second today. For the time being uh pluggables 400g is the right current market. Tomorrow 800 G. So the vendors will be probably keeping delivering those.

Mike Kelly:

Yeah, the pluggables you're talking about. Yeah, you're right. If you include pluggables into this discussion, that's already high volume, right? So but when you're talking about co-packaged on in a large GPU processor or CPU, that's that's different. And that is still is still low volume.

Francoise von Trapp:

Okay. So is it safe to say that we will be having similar conversations about this in five years?

John Knickerbocker:

Or in five years so I think the the number of optical connections per module and and the bandwidth is gonna climb over that time period. Yeah. But I think uh it's gonna take um, you know, pull from the industry uh to you know climb that curve. So as any new technology goes from smaller volumes and specialty, you start to get more standards created and more competition and more volume as the uh the demand goes up and and that drives itself in a business cycle to, you know, winner is the one that gives them a a cost-effective solution. And and so that's part of you know, not just the technology, but the business cycle to support a system requirements that's cost effective and can be met in the volumes of supply chain that are needed. And so all those things, like what's the exact timing? Good question. Um but I think over the next five years, if that's the time horizon that that uh is being discussed, that uh the the this technology is gonna advance substantially in that time period and beyond. And so so again, I think you're gonna be talking um much higher bandwidth, uh uh you know power, I think, um will come down, but it may be secondary to start with and then later become a more important driver. Uh so so again, I think all these things are gonna advance, and so it's actually very exciting times uh because I see the you know the the this topic area is just caught fire in the industry, and and so you've got many people working on it, and I think it's gonna uh you know go at a faster pace.

Francoise von Trapp:

Well, then I think it's gonna be great to continue having this conversation and and following this, I'm sure it'll be the topic of many future IMAPs. Well, thank you, all of you, for joining me today.

John Knickerbocker:

Thank you. Thank you.

Francoise von Trapp:

Next time on the 3D Insights podcast, we've got more from IMAP Symposium 2025. You'll hear from one newcomer to the industry about why he is interested in the field, and also a group of students talk about their experience at this year's IMAP. There's lots more to come, so tune in next time to the 3D Insights podcast. The 3D Insights Podcast is a production of 3D Insights LLC.