I got to talk with Lea Rosema this week! She’s an incredibly talented digital artist and front-end developer. She does quite a bit of art with some of the trickiest web technology out there: shaders! But rather than just learn it and use it, she helps other people learn and get more out of it. Several times that has taken the shape of Web Components. For example, a <shader-art> Web Component that takes some of the boilerplate work out of designing with them. See this Collection of examples.
Time Jumps
- 00:44 Guest introduction
- 03:38 Shaders
- 07:39 How does the shader work?
- 13:30 Sponsor: JS Nation
- 14:15 Dealing with strange looking code languages
- 17:53 What drew you to working in this tech?
- 20:20 Slider plugin
- 23:53 Half tone circle Pen
- 26:12 Magic Pixels
- 27:17 State of my brain Pen
- 28:37 Wandering through a twilight landscape Pen
Sponsor: JS Nation
Evan You, Kyle Simpson, Addy Osmani, Sarah Drasner – these are just a few of the speakers coming to this year’s JSNation, a 2-day conference focusing exclusively on JavaScript development. Discover the future of the JavaScript ecosystem and get connected to its stellar crowd! The format of the event will be hybrid. The first day (June 16) will be streamed from the Amsterdam venue. The second day (June 20) & numerous free workshops will be streamed to the global audience online.
Would you like to participate? Get 10% off on remote & in-person tickets with our discount code CodePen.
Transcript
[Radio channel adjustment]
Announcer: Today, on CodePen Radio.
Chris Coyier: Hello, everyone. Time for another CodePen Radio. This is 368, and I have an absolute CodePen must follow for all of you. It's Lea Rosema. How ya doin', Lea? Thanks for joining me on CodePen Radio.
Lea Rosema: Hi. Nice to see you.
[Laughter]
Chris: Yeah, that's great. I know this is an audio podcast but, as we record, we get to do video. You're sitting in apparently a new to you coworking space there in Hamburg, Germany, right?
Lea: Yeah, it's in the middle of the center of Hamburg. Yes, it's very nice. It's a historical building near the Rodingsmartkt. Yeah, it's a really nice location over here. There was a happy hour, and they had a lot of margaritas upstairs.
Chris: [Laughter] Nice.
Lea: [Laughter] Yeah. It was--
Chris: Well, I heard you're celebrating because it was a good week for you and you've just started a new job. Is that right?
Lea: Yeah. Yeah. Exactly. Yeah, in the start of the week, I was at headquarters in Shurgard, so I had a nice travel via trains through the whole of Germany, so from the very north of Hamburg to the very south Shurgard.
Chris: Mm-hmm.
Lea: Yeah. It was a nice trip to it, and there we had two welcome days where we learned everything about the company and, yeah, everything organizational. I got my laptops and stuff and also learned about the company culture and stuff.
Chris: What's the company? Can you tell us?
Lea: Yeah. It's Novatec GmbH. You may have heard of it, maybe when you follow Anna McDougall, who is an opera singer who switched to full-stack JavaScript development.
Chris: Wow!
Lea: She's gone viral. [Laughter] And she was the person who recommended me or connected me to the recruiting team, and we had a match.
Chris: Nice!
Lea: Yeah.
Chris: I'd love to see her get really mad at JavaScript and just, [sing at a high pitch]. [Laughter] That would be -- oh, I'm sure that's the most cliche joke that she's ever heard but I had to do it.
Okay. Well, great. So, your work -- I could talk to you about any number of things, I'm sure. But you're incredibly prolific on CodePen. Loads of creative work. And it sounds like you're very -- you just love it. Just love digital art and all the things around it, and you're just an all around front-end developer. I mean I hope that's still true. [Laughter]
Lea: [Laughter] Yeah, that's still true. Unfortunately, I didn't have much time in the last month because I had a lot of things to do with the recruiting stuff and getting the new job.
Chris: Right.
Lea: And interviews all the time. [Laughter]
Chris: Yeah, I suppose. I mean that's okay, though. You have a lifetime of work here to draw from. Maybe we could start with what's your most recent. I have to bring up shaders, because I'm just interested in them. They seem so exotic to me in this, like, I don't understand what I'm looking at, but usually the results from them are just outstandingly incredible.
If I was forced to (on this podcast) describe exactly what a shader even is, I'm not sure I could do it. So, maybe I could get you to do it.
Lea: [Laughter] Yeah, so the last thing I worked on was my shader Web component, which I put open-source on GitHub. Yeah, the last thing I did was implementing Web GL 2 support, so I can also use all the advantages of the new Web GL version.
Web GL 1 has a lot of some limitations, so like you can't loop through a loop where the end is not constant - things like that.
Chris: Mm-hmm.
Lea: There are also a lot of new features in Web GL 2 like 3D textures and, yeah, this is very exciting and also new for me to explore. Yeah, so my approach is building a little library and dabble around with it.
Chris: Yeah. What's the library? It's a Web component, meaning you use it. Is it the one that's digital-art? Is that the one, or is it a different one?
Lea: It is shader-art.
Chris: Shader-art. Okay.
Lea: You can NPM install shader-art, and then you have shader-art.register, and then you can use this shader art tech in your HTML.
Chris: Yeah.
Lea: Put some shader code in it and it renders the thing on Canvas. The canvas is created for you.
Chris: Oh, nice. So, it saves you some of that boilerplate where you have make a canvas and give it an ID and then select the canvas. These are the kinds of things the Web component is doing. What else does it help with, would you say?
Lea: It also has some plugin interface where you can plug in more features like there's a texture plugin where you can say, "Okay, I need a bunch of textures from maybe placekitten.com, and I want to use them in my shader.
There's also one ongoing thing I'm working on is the webcam plugin where I can use the webcam input and use them on the shader.
Yeah, there is also a uniform plugin where I can plug in .gui, which is the gui for setting values from JavaScript.
Chris: Yeah. Right.
Lea: And then it goes into the shader. [Laughter]
Chris: Oh, nice, because shader -- they're doing such creative stuff. It seems like the sky is the limit with what a shader can do. But it's doing stuff -- I don't know. You mentioned the texture plugin and Place Kitten.
Lea: Yeah.
Chris: It looks like, from the NPM docs, you can click over to an example that happens to be on CodePen -- that's pretty cool -- where it places an IMG tag of a Place Kitten. Then you, in the JavaScript, pull down the Web component, pull down the texture plugin, register the plugin, and then there's some magic in the HTML that I would love to understand a little more. Script tags with some pretty interesting looking JavaScript in it, and then the end result is this little kitten graphic with the pixels appear to be waving. You're like, "What is even happening?!"
It's an incredible result. How about we could talk about some of the JavaScript that makes it work. It does end up as a Canvas, ultimately, but it uses Web GL, which is part of a canvas. Can you walk me through how some of it even works?
Lea: Yeah, of course. One thing it basically does is, in Web GL, you always have to provide some kind of geometry, and I am, by default, providing a very simple geometry which is a plane with four coordinates just filling out the full canvas like minus 1, 1 as the top left corner of the screen.
Chris: Okay, so one, minus one is the top right?
Lea: Yes.
Chris: Okay. Meaning 00 is the middle, I guess?
Lea: Yeah, and with these coordinates, you need two triangles, basically. With these two triangles, I make up the plane. Yeah, this is geometry that I define automatically for the shader. Then this is a JavaScript part, so the JavaScript part is, here are the two triangles, and draw the two triangles with shaders.
Then, I use script text inside of Web components. One is the vertex shader. One is the fragment shader. The vertex shader can further process these geometry coordinates, so like apply some perspective to it - stuff like that.
Chris: Right.
Lea: By default, I just pass them on, so I do nothing with those coordinates in the vertex shader and just say, "Okay, these are my points for these two triangles." Then there's this fragment shader and the task of the fragment shader is to fill these two triangles with colors.
It's a crazy thing because this is the main function in this fragment shader, and this is called in parallel for each pixel.
Chris: Oh...
Lea: Massive, multiprocessing happening on the GPU. [Laughter]
Chris: Yeah. Okay, I didn't realize that. That main function runs on every pixel?
Lea: Yeah, this main function runs on every pixel on the canvas, yeah, in parallel, so highly parallel processing. Yeah, that's why it's so fucking fast. [Laughter]
Chris: Yeah. [Laughter] Yeah, that's amazing. You don't see any of the -- you don't see a for loop that loops over all the pixels. That's just implied - or something.
And the main function doesn't even take any parameters. It just knows what the fragment--
Lea: It has some parameters. You get the X-Y coordinate of the pixel.
Chris: Mm-hmm.
Lea: Yeah, then you can return a color dependent on this position. Then you can also pass multiple other things like maybe the mouse position or count up frame counter or use a time variable that is count up.
Chris: Yeah.
Lea: Yeah. This is how animating things work in this shader component. There's also a time uniform variable provided automatically via the shader art component, which you can use.
Chris: Oh... That helps with movement and timing. Yeah?
Lea: Yes. Yes, like building some simple animations. Yeah, like apply a size to it and then you have linking stuff.
Chris: Well, that's cool. Yeah, I think maybe what looks inscrutable about it is some of that -- it looks like it's typed, too, like if you're going to have a float.
Lea: Yep.
Chris: The statements are like float-y, which you'd see in literally C or Go or something.
Lea: Yeah.
Chris: Interesting.
Lea: Yeah, so the main struggle is it's exactly a lot like C or C++.
Chris: Mm-hmm.
Lea: You have float variables and it's everything typed. You have also some vector types. The awesome thing is it has some built in stuff like metrics multiplication. You don't have to code this yourself, so this is just built in, like cross product, .product, metrics multiplication. Stuff like this is built in, inside this language, and you can just use it and don't care about it.
Chris: Just use it. That's funny that it's actually -- the language is designed to be easy to use despite when you look at the code. It's a little bit like, "Wow! What is happening?" It's actually doing a lot for you. That's cool.
Lea: Yeah. Yeah, the only thing struggled a lot with is semicolons. They are mandatory in this language. Yeah, this is why I always put semicolons also in my JS code because if I wouldn't then I would forget them in my GL ... code.
Chris: Oh, gosh. That's wild.
Lea: [Laughter]
[Guitar music starts]
Chris: This episode of CodePen Radio is brought to you by JSNation Conference 2022. Evan You, Kyle Simpson, Addy Osmani, Sarah Drasner are just a few of the speakers coming this year to JS Nation, a two-day conference focusing exclusively on the JavaScript ecosystem. Super cool.
It's a hybrid format, so it's both remote and in-person - whatever you want to do. If you're going in-person, it's in Amsterdam. If you're going in remote, you can go from anywhere in the world that you want.
Now, go buy a ticket for it. It's coming up. June 16th is the first day, so watch for that. It's about a month away - maybe a little less as you're listening to this. If you buy a ticket, use the coupon code CODEPEN to get 10% off.
Meet the stars of the JavaScript scene by going to JSNation. Thanks so much for the support. It's going to be awesome.
[Guitar music ends]
Chris: I've been trying to learn Go. Every day, do little exercises and learn how we're using it at CodePen. Write network functions in it and stuff. It doesn't use semicolons, even though, in our JavaScript, we do all the time. But it doesn't mean there's no semicolons in the language. Sometimes the language uses it. Yeah. It messes with my brain.
Fortunately, when I'm writing in VS Code, the autoformatting of it takes over and generally does a good job of it. Yeah, it's wild though. It had me thinking. Looking at this code, it had me thinking about all these other.
I'm not very comfortable in Go, but sometimes I have to write a RegEx or look at a bash script or look at NGINX config. I was like, man, there are so many different languages that we sometimes have to deal with as Web developers that are so strange. SVG filters and even the SVG syntax altogether, or just the D syntax in SVG, how it can draw curves and stuff is so strange.
Is it kind of fun? You've done enough of these shaders that, I'm sure looking at most shaders, it doesn't look strange for you anymore.
Lea: It's kind of normal by now, so it still requires some thinking, like, "Okay." Yeah, but I always need, yeah, always use resources like The Book of Shaders by Patricio Gonzalez Vivo. Awesome resource when you want to look up certain functions like there are some built-in functions inside the shader language like a step function, which is very important to reduce ifs inside the shader code.
Chris: Okay.
Lea: You can look at every GLSL function inside The Book of Shaders. there is a glossary, and you can just look all the things up. There's also something like a graph which tells you, "Okay, when you put in this value, you get that out of it." This is just awesome, the site.
Chris: Nice.
Lea: Yeah, what I also use a lot is resources by Inigo Quilez. He also has a YouTube channel where he explains all the math behind the arts he does. There's a great explanation about how drawing circles with the shader work or drawing several -- all kinds of shades like line segments. He also has great videos about it where he explains it all how the math behind the work.
Yeah. I also got math training, so like you don't have to be good at math to start at shaders. But you get better while doing it.
Chris: Yeah, I bet. I bet. You have so many of these examples. It's so cool. I'm sure other people would say, "Oh, I look at Lea's examples." [Laughter] "That's how I get started."
Where do the ideas come? At a higher level, what attracts you to working with that particular medium? I know that's all you do. You have all kinds of other examples, too. Is it just a desire to use that particular technology, or does it allow you to express an idea in the way that you want to?
Lea: It's mostly a fun idea. It's a form of relaxing, thinking up--
Chris: Really?!
Lea: Thinking up stuff and, yeah, it's a form of relaxing for me, so like, yeah, thinking out different kinds of things like also trying out new things like what I also really liked was building the space, sky with shaders using noise. There's really cool things. You can do all kinds of graphics just by using a noise function.
Chris: Noise. Mm-hmm.
Lea: It just makes it so awesome. [Laughter]
Chris: Yeah. I get you. That stuff can really be fun. Although, I'm not sure I'm at the level where this type of work would feel relaxing to me, but there are other types of coding work for me that do feel very - I don't know - chill in that way. I wish I could give that gift to people because it's so--
Clearly, it engages your brain and it opens doors for you. It makes you think of other creative things you want to do. It turns you into a maker, a little bit.
I get that some people just want to sit down and watch TV or something. No shade on watching TV. I watch TV all the time. That's a nice way to relax too. But if that's all you ever do to relax, I feel like it's not -- you're not getting as much out of life as you could be if, even in your relaxing time, using his brain as much as you do.
Lea: [Laughter] I also like to go for walks. [Laughter]
Chris: Yeah. [Laughter] Right. Not saying you have to be productive at all seconds, but it's cool to be able to do something with your brain that also feels relaxing.
I was just in a Discord chat with somebody who was looking at a slider plugin. It was called Splide, splidejs.com. It's neat. There are a million carousel plugins in the world, but sometimes there are some that are better than others or use more modern technologies than others do. They seem to come and go - you know.
This one, one of the ways it looks like they're trying to monetize it is where one of the transitions between the slides in the carousel is a shader. I'll link it in the Riverside chat there. It looks so neat.
You know I only just bring it up to see if you had any thoughts on it or anything. But it's interesting when you see a shader be like an upsell. It's such an interesting idea that you could use its interestingness as a way to make money. [Laughter] I don't know. It's cool.
Lea: It's nice. [Laughter] I must have a look at it.
Chris: Yeah, it's like one image to another and they melt into each other. It's so cool.
Lea: Yeah, I was also thinking of experimenting more towards some page transition stuff.
Chris: Right.
Lea: Yeah, also working on some kind of, yeah, relaunch for my website and build some showcase. Yeah, I also wanted to build some transitions into it. Maybe, yeah, some carrousel ... like this, maybe.
Chris: Maybe, yeah.
Lea: Yeah.
Chris: Can you do it with the DOM? I get that you can feed an image to Canvas. But can you feed typography, like a div, to Web GL in which to do its stuff on?
Lea: Yeah. That's really a bit difficult because everything you have to provide to Web GL needs to be in an image format. So, yeah, that's quite difficult.
Chris: Oh, I see. There's no snapshot API in the browser, really. You'd be like, "Give me an image of this div," is not really a thing, right?
Lea: Yeah. Yeah, there are some -- yeah, some ways are maybe you use vector formats.
Chris: Oh, right. Yeah, maybe. I think of like Puppeteer, how you can get an image out of that, but that doesn't really run in the browser.
Lea: Yeah. My approach was basically also to use something like Puppeteer to make a bunch of screenshots from my work, and basically use them as carousel images. Yeah, also one thing I considered was this Web component shader art and apply some CSS 3D transforms to it.
Chris: Oh, wow!
Lea: This could also work, maybe.
Chris: Right. Oh, gees.
Lea: I have to try this. [Laughter]
Chris: Yeah, you should try that because you think of, okay, you do everything in Canvas, but I never really thought of then taking the canvas and applying additional stuff on top of it with CSS. But why not? It would work.
Lea: Yeah.
Chris: That's cool. That's cool. It looks like -- I don't know. I'm just looking at some of my favorites that I bookmarked here. You have one from--
Sorry, I don't mean to blindside you with old Pens that aren't loaded up in your brain or anything at the moment. But you did one with half tone circles.
Lea: Yeah.
Chris: That's so cool. It's Canvas, but it's using GreenSock inside of it. Nice reminder that GreenSock works on all kinds of stuff, including Web GL, apparently.
Lea: Yeah, you can use the scroll wheel to change the image of the animation.
Chris: Yeah. Now that's digital art, if you ask me. It's just beautiful just to watch. If it was on a big monitor in an arena, you'd be like, "Wow, that's a beautiful piece of art." But because it's on the Web and we have other inputs, I think it's so cool to be thinking about those. In this case, yeah, you used the scroll wheel to have it be the interaction mechanism for this piece of art. It really takes it to another level, I think.
Lea: I also like to have the simple library tool. Yeah. It just works on CodePen and it's easy to use.
Chris: Yeah. It takes down the time from - I don't know - idea to satisfaction. [Laughter]
Lea: [Laughter] Yeah. Maybe it's not the use friendliest thing. [Laughter] You still have to do more shader stuff.
Chris: Right.
Lea: I think I can also improve the documentation. Yeah.
Chris: Right. I wonder if you'll convert some of your other ideas. There are a lot of things. There's an awful lot of shader-based stuff that you've built that's not using this, right?
Lea: Yeah. Yeah.
Chris: Yeah.
Lea: I also wrote another library which is called GLEA [laughter], which is based on my name but also based on GL.
Chris: Right.
Lea: Yeah.
Chris: That was just like a precursor to the shader one?
Lea: Yeah. It was just trying things out, so I don't have an opinion which is a better approach. Both work fine.
I also did a third Web GL library which is called Magic Pixels. It was an approach to refactor how Three.js works.
Chris: Whoa.
Lea: So, like, "Okay, build some more complex geometries like make up a sphere or make up a box with many intersections."
Chris: Whoa.
Lea: You can then apply very much deformations on the dice when you render this thing in 3D and also provide things like perspective camera and, yeah, just see how the Three.js thing was made. Yeah. Yeah, this was the Magic Pixels thing that I built.
Chris: Wow. You're at least three helper Web components deep. Probably more than that. I like it though.
There's one. For example, I think the Pen that's kind of pinned on your profile is The State of My Brain.
Lea: [Laughter] Yeah.
Chris: Which is -- yeah. It's like fractals and rainbows and all kinds of stuff, and you can control it, which is one of my favorite things on CodePen is when people make digital art and then give you controls. It looks like these controls you kind of built yourself rather than using DAT GUI, knobs, or anything. But it's great.
When you look at that now, would you be like, "Oh, I should rewrite that with my Web component"?
Lea: No, I'm still very happy with it. The only thing is that The State of My Brain, at that time there was a real mess going on in my brain. [Laughter]
Chris: Oh, it's real art. It was really a reflection of how you were feeling at the time.
Lea: Yeah. Yeah. I had a very difficult time back then. Yeah, I also used writing this shader program to chill down and relax. Yeah, so to go on.
Chris: Wow. That's really meaningful, though. I know you probably don't want to think back on dark times, and I don't want to make you do that, but that's what art is. You know?
Lea: Yeah.
Chris: It's beautiful, in a way. Then here -- oh, gosh. There are so many of these that I just think are so beautiful.
Possibly my favorite. I don't know what attracts me to it so much. But there's this one called Wandering Through a Generative Twilight Landscape where the background is these subtle gradients that are very calming. The whole thing is really calming, but the ground is very rough but not. It's smooth but has big bumps and hills and stuff.
Lea: Yeah.
Chris: Like a lot of -- what I think of this Web GL Pens is you're flying through it. You're experiencing. There are so many examples of this technology that's like that, and I kind of love it. The way that you can experience this is like you were there and it's like there's a camera. The camera is moving through this space.
But it's clearly randomized or at least it seems like it is. It's just so calming and good. Is there a story behind that one?
Lea: Is there a story behind that one? Um... yeah, I just wanted to try things out. Many of my Pens just use a fixed geometry, like just a rectangle filling up the canvas.
Chris: Mm-hmm.
Lea: This one was, I think, the first one where I did a bit more, so there's this plane with big bumps in it with these deformations happening and also animations, wandering animations. There's a second plane with this sky animation. Yeah, this was the first one where I did a little more geometry in it, so not just the plane.
Chris: Yeah. That makes sense. There are two different main animations happening, and they're at different places, but it all comes together in the end.
Lea: Also it was a practice for, yeah, applying animations to geometry through this vertex shader. Yeah, these deformations, these bumps in the plane are happening. They are not directly in the geometry. I don't change the bumps like this in the JavaScript code. I just use this vertex shader for it for randomizing it. Yeah.
Chris: Oh...
Lea: I'm tying these deformations to the ... axis via this vertex shader. Yeah, that's a very performant way to do it.
Chris: I bet. I bet. Yeah, it's not like you called math.random - or whatever. The deformations are a shader as well.
Lea: Yeah, it's all happening on the GPU, which makes it fast.
Chris: Yeah, and these things are so fast. I know they're known for that, but it's really outstanding to see animation that's this complex looking. You know computers never break a sweat. There's just something magical about the performance of it. I guess that's the power of the GPU, huh?
Lea: Yeah.
Chris: Yeah. Yeah, it's pretty wonderful. The black and white in this one just is so striking to me. You expect a sand worm from Beetlejuice to come out - or whatever - or a tent from "The Night Circus."
Anyway, thanks for talking to me, so much. You should probably get back to some margaritas. You can hear them back there partying.
Lea: [Laughter] Yeah.
Chris: [Laughter] But thanks for taking time to talk to me for CodePen Radio. It was a pleasure to talk to you. Really amazing, creative stuff.
Lea: Yeah. A pleasure to talk to you.
Chris: Wonderful. Well, I hope we get to see each other at some point in this crazy world. That would be awesome to cross paths. But again, take care. Any last words for the people of CodePen?
Lea: Yeah. Just remember to have fun, I think, and relax. Relax and code.
Chris: Yeah. All right. Wonderful. Okay. See you later.
Lea: See you.
[Radio channel adjustment]