Meaningful Transformations

We discovered the work of Tom Lechner for the first time at the Libre Graphics Meeting 2010 in Brussels. Tom traveled from Portland to present Laidout, an amazing tool that he made to produce his own comic books and also to work on three dimensional mathematical objects. We were excited about how his software represents the gesture of folding, loved his bold interface decisions plus were impressed by the fact that Tom decided to write his own programming framework for it. A year later, we met again in Montreal, Canada for the Libre Graphics Meeting 2011 where he presents a follow-up. With Ludivine Loiseau ¹ and Pierre Marchand ², we finally found time to sit down and talk.

Well, Laidout is software that I wrote to lay out my cartoon books in an easy fashion. Nothing else fit my needs at the time, so I just wrote it.

It works for any image, basically, and gradients. It does not currently do text. It is on my todo list. I usually write my own text, so it does not really need to do text. I just make an image of it.

But that's all images too. I guess it's two forms of laying out. It's laying out pieces of paper that remain whole in themselves, or you can take an image and lay it out on smaller pieces of paper. Tiling, I guess you could call it.

OK. So, you need a pattern. I had just a shirt that sort of fit and I approximated it on a big piece of paper, to figure out what the pieces were shaped like, and took a photograph of that. I used a perspective tool to remove the distortion. I had placed rulers on the ground so that I could remember the actual scale of it. Then once it was in the computer, I traced over it in Inkscape, to get just the basic outline so that I could manipulate further. Blender didn't want to import it so I had to retrace it. I had to use Blender to do it because that lets me shape the pattern, take it from flat into something that actually makes 3D shapes so whatever errors were in the original pattern that I had on the paper, I could now correct, make the sides actually meet and once I had the molded shape, and in Blender you have to be extremely careful to keep any shape, any manipulation that you do to make sure your surface is still unfoldable into something flat. It is very easy to get away from flat surfaces in Blender. Once I have the molded shape, I can export that into an .off file which my unwrapper can import and that I can then unwrap into the sleeves and the front and the back as well as project a panoramic image onto those pieces. Once I have that, it becomes a pattern laid out on a giant flat surface. Then I can use Laidout once again to tile pages across that. I can export into a .pdf with all the individual pieces of the image that were just pieces of the larger image that I can print on transfer paper. It took forty iron-on transfer papers I ironed with an iron provided to me by the people sitting in front of me so that took a while but finally I got it all done, cut it all out, sewed it up and there you go.

Could you say something about your interest in moving from 2D to 3D and back again? It seems everything you do is related to that?

I don't know. I've been making sculpture of various kinds for quite a long time. I've always drawn. Since I was about eighteen, I started making sculptures, mainly mathematical woodwork. I don't quite have access to a full woodwork workshop anymore, so I cannot make as much woodwork as I used to. It's kind of an instance of being defined by what tools you have available to you, like you were saying in your talk. I don't have a woodshop, but I can do other stuff. I can still make various shapes, but mainly out of paper. Since I had been doing woodwork, I picked up photography I guess and I made a ton of panoramic images. It's kind of fun to figure out how to project these images out of the computer into something that you can physically create, for instance a T-shirt or a ball, or other paper shapes.

Is there ever any work that stays in the computer, or does it always need to become physical?

Usually, for me, it is important to make something that I can actually physically interact with. The computer I usually find quite limiting. You can do amazing things with computers, you can pan around an image, that in itself is pretty amazing but in the end I get more out of interacting with things physically than just in the computer.

But with Laidout, you have moved folding into the computer! Do you enjoy that kind of reverse transformation?

It is a challenge to do and I enjoy figuring out how to do that. In making computer tools, I always try to make something that I can not do nearly as quickly by hand. It's just much easier to do in a computer. Or in the case of spherical images, it's practically impossible to do it outside the computer. I could paint it with airbrushes and stuff like that but that in itself would take a hundred times longer than just pressing a couple of commands and having the computer do it all automatically.

My feeling about your work is that the time you spent working on the program is in itself the most intriguing part of your work. There is of course a challenge and I can imagine that when you are doing it like the first time you see a rectangle, and you see it mimic a perspective you think wow I am folding a paper, I have really done something. I worked on imposition too but more to figure out how to work with .pdf files and I didn't go this way of the gesture like you did. There is something in your work which is really the way you wrote your own framework for example and did not use any existing frameworks. You didn't use existing GUIs and toolboxes. It would be nice to listen to you about how you worked, how you worked on the programming.

I think like a lot of artists, or creative people in general, you have to enjoy the little nuts and bolts of what you're doing in order to produce any final work, that is if you actually do produce any final work. Part of that is making the tools. When I first started making computer tools to help me in my artwork, I did not have a lot of experience programming computers. I had some. I did little projects here and there. So I looked around at the various toolkits, but everything seemed really rigid. If you wanted to edit some text, you had this little box and you write things in this little box and if you want to change numbers, you have to erase it and change tiny things with other tiny things. It's just very restrictive. I figured I could either figure out how to adapt those to my own purposes, or I could just figure out my own, so I figured either way would probably take about that same amount of time I guessed, in my ignorance. In the process, that's not quite been true. But it is much more flexible, in my opinion, what I've developed, compared to a lot of other toolkits. Other people have other goals, so I'm sure they would have a completely different opinion. For what I'm doing, it's much more adaptable.

I don't think I ever actually took computer programming classes. I grew up with a Commodore 64, so I was always making letters fly around the screen and stuff like that, and follow various curves. So I was always doing little programming tricks. I guess I grew up in a household where that sort of thing was pretty normal. I had two brothers, and they both became computer programmers. And I'm the youngest, so I could learn from their mistakes, too. I hope.

We can discuss at length about how actual toolkits don't match your needs, but in the end, you want to input certain things. With any recent toolkit, you can do that. It's not that difficult or time consuming. The way you do it, you really enjoy it, by itself. I can see it as a real creative work, to come up with new digital shapes.

I think it's definitely part of the work. That's kind of the nuts and bolts that you have to enjoy to get somewhere else. But if I look back on it, I spend a huge amount of time just programming and not actually making the artwork itself. It's more just making the tools and all the programming for the tools. I think there's a lot of truth to that. When it comes time to actually make artwork, I do like to have the tool that's just right for the job, that works just the way that seems efficient.

I think the program itself is an artwork, very much. To me it is also a reflection on moving between 2D and 3D, about physical computation. Maybe this is the actual work. Would you agree?

I don't know. To an extent. In my mind, I kind of class it differently. I've certainly been drawing more than I've been doing technical stuff like programming. In my mind, the artwork is things that get produced, or a performance or something like that. And the programming or the tools are in service to those things. That's how I think of it. I can see that... I've distributed Laidout as something in itself. It's not just some secret tool that I've put aside and presented only the artwork. I do enjoy the tools themselves.

I have a question about how the 2D imagines 3D. I've seen Pierre and Ludi write imposition plans. I really enjoy reading this, almost as a sort of poetry, about what it would be to be folded, to be bound like a book. Why is it so interesting for you, this tension between the two dimensions?

I don't know. Perhaps it's just the transformation of materials from something more amorphous into something that's more meaningful, somehow. Like in a book, you start out with wood pulp, and you can lay it out in pages and you have to do something to that in order to instil more meaning to it.

Somewhat. I've bound a few things by hand. Most of my cartoon books ended up being just stapled, like a stack of paper, staple in the middle and fold. Very simple. I've done some where you cut down the middle and lay the sides on top and they're perfect bound. I've done just a couple where it's an actual hand bound, hard cover. I do enjoy that. It's quite a time consuming thing. There's quite a lot of craft in that. I enjoy a lot of hand made, do-it-yourself activities.

I guess that's kind of my goal. I did look up classic book binding techniques and how people do it and what sort of problems they encounter. I'm not sure if I've encompassed everything in that, certainly. But just the basics of folding and trimming, I've done my best to be able to do the same sort of techniques that have been done in the past, but only manually. The computer can remember things much more easily.

Imposition plans are quite fixed, you have this paper size and it works with specific imposition plans. I like the way your tool is very organic, you can play with it. But in the end, something very classic comes out, an imposition plan you can use over and over, which gives a sort of continuity.

What's impressive is the attention you put into the visualization. There are some technical programs which do really big imposition stuff, but it's always at the printer. Here, you can see the shape being peeled. It's really impressive. I agree with Femke that the program is an artwork too, because it's not only technical, it's much more.

So, far not really completely. When you fold, you introduce slight twists and things like that. And that depends on the stiffness of the paper and the thickness of the paper and I've not adequately dealt with that so much. If you just have one fold, it's pretty easy to figure out what the creep is for that. You can do tests and you can actually measure it. That's pretty easy to compensate for. But if you have many more folds than that, it becomes much more difficult.

That would be very interesting. To imagine paper in digital space, to give an idea of what might come out in the end. Then you really have to work your metaphors, I think?

A long time ago, I did a lot of T-shirt printing. Something that I did not particularly have was a way to visualize your final image on some kind of shirt and the same thing applies for book binding, too. You might have a strange texture. It would be nice to be able to visualize that beforehand, as well as the thickness of the paper that actually controls physical characteristics. These are things I would like to incorporate somehow but haven't gotten around to.

You talked about working with physical input, having touchpads... Can you talk a bit more about why you're interested in this?

You can do a lot of things with just a mouse and a keyboard. But it's still very limiting. You have to be sitting there, and you have to just control those two things. Here's your whole body, with which you can do amazing things, but you're restricted to just moving and clicking and you only have a single point up on the screen that you have to direct very specifically. It just seems very limiting. It's largely an unexplored field, just to accept a wider variety of inputs to control things. A lot of the multitouch stuff that's been done is just gestures for little tiny phones. It's mainly for browsing, not necessarily for actual work. That's something I would like to explore quite a lot more.

There's tons of sci fi movies, like 'Minority Report', where you wear these gloves and you can do various things. Even that is still just mainly browsing. I saw one, it was a research project by this guy at Caltech. He had made this table and he wore polarized glasses so he could look down at this table and see a 3D image. And then he had gloves on, and he could sculpt things right in the air. The computer would keep track of where his hand is going. Instead of sculpting clay, you're sculpting this 3D mesh. That seemed quite impressive to me.

It's something that's on my mind. I just got something called the Eggbot. You can hold spheres in this thing and it's basically a plotter that can print on spherical surfaces or round surfaces. That's something I'd like to explore some more. I've made various balls with just my photographic panoramas glued onto them. But that could be used to trace an outline for something and then you could go in with pens or paints and add more detail. If you're trying to paint on a sphere, just paint and no photograph, laying out an outline is perhaps the hardest part. If you simplify it, it becomes much easier to make actual images on spheres. That would be fun to explore.

I'd like to come back to the folding. Following your existing aesthetic, the stiffness and the angles of the drawing are very beautiful. Is it important you, preserving the aesthetic of your programs, the widgets, the lines, the arrows...

I think the specific widgets, in the end, are not really important to me at all. It's more just producing an actual effect. So if there is some better way, more efficient way, more adaptable way to produce some effect, then it's better to just completely abandon what doesn't work and make something that's new, that actually does work. Especially with multitouch stuff, a lot of old widgets make no more sense. You have to deal with a lot of other kinds of things, so you need different controls.

It makes sense, but I was thinking about the visual effect. Maybe it's not Laidout if it's done in Qt.

Your visuals and drawings are very aesthetically precise. We're wondering about the aesthetics of the program, if it's something that might change in the future.

That's an interesting question as well. But particularly the interface, it's very related to your drawings. There's a distinct quality. I was wondering how you feel about that, how the interaction with the program relates to the drawings themselves.

I think it just comes back to being very visually oriented. If you have to enter a lot of values in a bunch of slots in a table, that's not really a visual way to do it. Especially in my artwork, it's totally visual. There's no other component to it. You draw things on the page and it shows up immediately. It's just very visual. Or if you make a sculpture, you start with this chunk of stuff and you have to transform it in some way and chop off this or sand that. It's still all very visual. When you sit down at a computer, computers are very powerful, but what I want to do is still very visually oriented. The question then becomes: how do you make an interface that retains the visual inputs, but that is restricted to the types of inputs computers need to have to talk to them?

The way someone sets up his workshop says a lot about his work. The way you made Laidout and how you set up its screen, it's important to define a spot in the space of the possible.

What is nice is that you made the visualisation so important. The windows and the rest of the interface is really simple, the attention is really focused on what's happening. It is not like shiny windows with shadows everywhere, you feel like you are not bothered by the machine.

At the same time, the way you draw the thickness of the line to define the page is a bit large. For me, these are choices, and I am very impressed because I never manage to make choices for my own programs. The programs you wrote, or George Williams, make a strong aesthetic assertion like: This is good. I can't do this. I think that is really interesting.

Heavy page borders, that still comes down to the visual thing you end up with, is still the piece of paper so it is very important to find out where that page outline actually is. The more obvious it is, the better.

Yes, I think it makes sense. For a while now, I paid more attention than others in Scribus to these details like the shape of the button, the thickness of the lines, what pattern do you chose for the selection, etcetera. I had a lot of feedback from users like: I want this, this is too big and at some point you want to please everybody and you don't make choices. I don't think that you are so busy with what others think.

Not that I know of (laughter). I know that there is at least one other person that actually used it to produce a booklet. So I know that it is possible for someone other than myself to make things with it. I've gotten a couple of patches from people to not make it crash at various places but since Laidout is quite small, I can just not pay any attention to criticism. Partially because there isn't any, and I have particular motivations to make it work in a certain way and so it is easier to just go forward.

I think people that want to use your program are probably happy with this kind of visualisation. Because you wrote it alone, there is also a consistency across the program. It is not like Scribus, that has parts written by a lot of people so you can really recognize: this is Craig (Bradney), this is Andreas (Vox), this is Jean (Ghali), this is myself. There is nothing to follow.

I remember Donald Knuth talking about TeX and he was saying that the entire program was written from scratch three times before its current incarnation. I am sympathetic to that style of programming.

I think it is a good idea, to start again. To come back to a little detail. Is there a fileformat for your imposition tool, to store the imposition plan? Is it a text or a binary format?

It is text-based, an indented file format, sort of like Python. I did not want to use XML, every time I try to use XML there are all these greater thans and less thans. It is better than binary, but it is still a huge mess. When everything is indented like a tree, it is very easy to find things. The only problem is to always input tabs, not spaces. I have two different imposition types, basically, the flat-folding sheets and the three dimensional ones. The three dimensional one is a little more complicated.

Not exactly. It lists what folds exists. If you have a five by five grid, it will say Fold along this line, over in such and such direction. What it actually translates to in the end, is not currently stored in the file. Once you are in Laidout you can export into a PodofoImpose plan file.

Does it talk about turning pages? This I find beautiful in PodofoImpose plans, you can almost follow the paper through the hands of the program. Turn now, flip backwards, turn again. It is an instruction for a dance.

The text you can read in the PodofoImpose plans was taken from what Ludi and me did by hand. One of us was folding the paper, and the other was writing it into the plan. I think a lot of the things we talk about, are putting things from the real world into the computer. But you are putting things from the computer into the real world.

Can you describe again these two types of imposition, the first one being very familiar to us. It must be the most frequently asked question on the Scribus mailing list: How to do imposition. Even the most popular search term on the OSP website is 'Bookletprinting'. But what is the difference with the plan for a 3D object? A classic imposition plan is also somehow about turning a flat surface into a three dimensional object?

It is almost translatable. I'm reworking the 3D version to be able to incorporate the flat folding. It is not quite there yet, the problem is the connection between the pages. Currently, in the 3D version, you have a shape that has a definitive form and that controls how things bleed across the edges. When you have a piece of paper for a normal imposition, the pages that are next to each other in the physical form are not necessarily related to each other at all in the actual piece of paper. Right now, the piece of paper you use for the 3D model is very defined, there is no flexibility. Give me a few months!

It is a different approach. One person wanted to do flexagons, it is sort of like origami I guess, but it is not quite as complicated. You take a piece of paper, cut out a square and another square, and than you can fold it and you end up with a square that is actually made up of four different sections. Than you can take the middle section, and you get another page and you can keep folding in strange ways and you get different pages. Now the question becomes: how do you define that page, that is a collection of four different chunks of paper? I'm working on that!

We talk about the move from 2D to 3D as if these pages are empty. But you actually project images on them and I keep thinking about maps, transitional objects where physical space is projected on paper which then becomes a second real space and so on. Are you at all interested in maps?

A little bit. I don't really want to because it is such a well-explored field already. Already for many hundreds of years the problem is how do you represent a globe onto a more or less two dimensional surface. You have to figure out a way to make globe gores or other ways to project it and than glue it on to a ball for example. There is a lot of work done with that particular sort of imagery, but I don't know.

Yes. One thing that might be interesting to do though is when you have a ball that is a projection surface, then you can do more things, like overlays onto a map. If you want to simulate earthquakes for example. That would be entertaining.

For the first couple that I made, I made this 30-sided polyhedron that you could mount a camera inside and it sat on a base in a particular way so you could get thirty chunks of images from a really cheap point and shoot camera. You do all that, and you have your thirty images and it is extremely laborious to take all these thirty images and line them up. That is why I made the 3D portion of Laidout, it was to help me do that in an easier fashion. Since then I've got a fish-eyed lens which simplifies things quite considerably. Instead of spending ten hours on something, I can do it in ten minutes. I can take 6 shots, and one shot up, one shot down. In Hugin you can stitch them all together.

And the kinds of things you photograph? We saw the largest rodent on earth? How do you pick a spot for your images?

I am not really sure. I wander around and than photograph whatever stands out. I guess some unusual configuration of architecture frequently or sometimes a really odd event, or a political protest sometimes. The trick with panoramas is to find an area where something is happening all over the globe. Normally, on sunny days, you take a picture and all your image is blank. As pretty as the blue sky is, there is not a lot going on there particularly.

Panoramic images are usually spherical or circular. Do you take certain images with a specific projection surface in mind?

To an extent. I take enough images. Once I have a whole bunch of images, the task is to select a particular image that goes with a particular shape. Like cubes there are few lines and it is convenient to line them up to an actual rectangular space like a room. The tetrahedron made out of cones, I made one of Mount St. Helens, because I thought it was an interesting way to put the two cones together. You mentioned 3D printers earlier, and one thing I would like to do is to extend the panoramic image to be more like a progression. For most panoramic images, the focal point is a single point in space. But when you walk along a trail, you might have a series of photographs all along. I think it could be an interesting work to produce, some kind of ellipsoidal shape with a panoramic image that flows along the trail.

Back to Laidout, and keeping with the physical and the digital. Would there be something like a digital papercut?

In a sense there is. In the imposition view, right now I just have a green bar to tell where the binding is. However when you do a lot of folds, you usually want to do a staple. But if you are stapling and there is not an actual fold there, than you are screwed.

Meaningful transformations