Many software projects emerge because—somewhere out there—a programmer had a personal problem to solve.
That’s more or less what happened to Graydon Hoare. In 2006, Hoare was a 29-year-old computer programmer working for Mozilla, the open-source browser company. Returning home to his apartment in Vancouver, he found that the elevator was out of order; its software had crashed. This wasn’t the first time it had happened, either.
Hoare lived on the 21st floor, and as he climbed the stairs, he got annoyed. “It’s ridiculous,” he thought, “that we computer people couldn’t even make an elevator that works without crashing!” Many such crashes, Hoare knew, are due to problems with how a program uses memory. The software inside devices like elevators is often written in languages like C++ or C, which are famous for allowing programmers to write code that runs very quickly and is quite compact. The problem is those languages also make it easy to accidentally introduce memory bugs—errors that will cause a crash. Microsoft estimates that 70% of the vulnerabilities in its code are due to memory errors from code written in these languages.
Most of us, if we found ourselves trudging up 21 flights of stairs, would just get pissed off and leave it there. But Hoare decided to do something about it. He opened his laptop and began designing a new computer language, one that he hoped would make it possible to write small, fast code without memory bugs. He named it Rust, after a group of remarkably hardy fungi that are, he says, “over-engineered for survival.”
Seventeen years later, Rust has become one of the hottest new languages on the planet—maybe the hottest. There are 2.8 million coders writing in Rust, and companies from Microsoft to Amazon regard it as key to their future. The chat platform Discord used Rust to speed up its system, Dropbox uses it to sync files to your computer, and Cloudflare uses it to process more than 20% of all internet traffic.
When the coder discussion board Stack Overflow conducts its annual poll of developers around the world, Rust has been rated the most “loved” programming language for seven years running. Even the US government is avidly promoting software in Rust as a way to make its processes more secure. The language has become, like many successful open-source projects, a barn-raising: there are now hundreds of die-hard contributors, many of them volunteers. Hoare himself stepped aside from the project in 2013, happy to turn it over to those other engineers, including a core team at Mozilla.
It isn’t unusual for someone to make a new computer language. Plenty of coders create little ones as side projects all the time. But it’s meteor-strike rare for one to take hold and become part of the pantheon of well-known languages alongside, say, JavaScript or Python or Java. How did Rust do it?
To grasp what makes Rust so useful, it’s worth taking a peek beneath the hood at how programming languages deal with computer memory.
You could, very crudely, think of the dynamic memory in a computer as a chalkboard. As a piece of software runs, it’s constantly writing little bits of data to the chalkboard, keeping track of which one is where, and erasing them when they’re no longer needed. Different computer languages manage this in different ways, though. An older language like C or C++ is designed to give the programmer a lot of power over how and when the software uses the chalkboard. That power is useful: with so much control over dynamic memory, a coder can make the software run very quickly. That’s why C and C++ are often used to write “bare metal” code, the sort that interacts directly with hardware. Machines that don’t have an operating system like Windows or Linux, including everything from dialysis machines to cash registers, run on such code. (It’s also used for more advanced computing: at some point an operating system needs to communicate with hardware. The kernels of Windows, Linux, and MacOS are all significantly written in C.)
But as speedy as they are, languages like C and C++ come with a trade-off. They require the coder to keep careful track of what memory is being written to, and when to erase it. And if you accidentally forget to erase something? You can cause a crash: the software later on might try to use a space in memory it thinks is empty when there’s really something there. Or you could give a digital intruder a way to sneak in. A hacker might discover that a program isn’t cleaning up its memory correctly—information that should have been wiped (passwords, financial info) is still hanging around—and sneakily grab that data. As a piece of C or C++ code gets bigger and bigger, it’s possible for even the most careful coder to make lots of memory mistakes, filling the software with bugs.
“In C or C++ you always have this fear that your code will just randomly explode,” says Mara Bos, cofounder of the drone firm Fusion Engineering and head of Rust’s library team.
In the ’90s, a new set of languages like Java, JavaScript, and Python became popular. These took a very different approach. To relieve stress on coders, they automatically managed the memory by using “garbage collectors,” components that would periodically clean up the memory as a piece of software was running. Presto: you could write code that didn’t have memory mistakes. But the downside was a loss of that fine-grained control. Your programs also performed more sluggishly (because garbage collection takes up crucial processing time). And software written in these languages used much more memory. So the world of programming became divided, roughly, into two tribes. If software needed to run fast or on a tiny chip in an embedded device, it was more likely to be written in C or C++. If it was a web app or mobile-phone app—an increasingly big chunk of the world of code—then you used a newer, garbage-collected language.
With Rust, Hoare aimed to create a language that split the difference between these approaches. It wouldn’t require programmers to manually figure out where in memory they were putting data; Rust would do that. But it would impose many strict rules on how data could be used or copied inside a program. You’d have to learn those coding rules, which would be more onerous than the ones in Python or JavaScript. Your code would be harder to write, but it’d be “memory safe”—no fears that you’d accidentally inserted lethal memory bugs. Crucially, Rust would also offer “concurrency safety.” Modern programs do multiple things at once—concurrently, in other words—and sometimes those different threads of code try to modify the same piece of memory at nearly the same time. Rust’s memory system would prevent this.
When he first opened his laptop to begin designing Rust, Hoare was already a 10-year veteran of software, working full time at Mozilla. Rust was just a side project at first. Hoare beavered away at it for a few years, and when he showed it to other coders, reaction was mixed. “Some enthusiasm,” he told me in an email. “A lot of eye-rolls and ‘This will never work’ or ‘This will never be usable.’”
Executives at Mozilla, though, were intrigued. Rust, they realized, could help them build a better browser engine. Browsers are notoriously complex pieces of software with many opportunities for dangerous memory bugs.
One employee who got involved was Patrick Walton, who’d joined Mozilla after deciding to leave his PhD studies in programming languages. He remembers Brendan Eich, the inventor of JavaScript, pulling him into a meeting at Mozilla: “He said, ‘Why don’t you come into this room where we’re going to discuss design decisions for Rust?’” Walton thought Rust sounded fantastic; he joined Hoare and a growing group of engineers in developing the language. Many, like Mozilla engineers Niko Matsakis and Felix Klock, had academic experience researching memory and coding languages.
In 2009, Mozilla decided to officially sponsor Rust. The language would be open source, and accountable only to the people making it, but Mozilla was willing to bootstrap it by paying engineers. A Rust group took over a conference room at the company; Dave Herman, cofounder of Mozilla Research, dubbed it “the nerd cave” and posted a sign outside the door. Over the next 10 years, Mozilla employed over a dozen engineers to work on Rust full time, Hoare estimates.
“Everyone really felt like they were working on something that could be really big,” Walton recalls. That excitement extended outside Mozilla’s building, too. By the early 2010s, Rust was attracting volunteers from around the world, from every nook of tech. Some worked for big tech firms. One major contributor was a high school student in Germany. At a Mozilla conference in British Columbia in 2010, Eich stood up to say there’d be a talk on an experimental language, and “don’t attend unless you’re a real programming language nerd,” Walton remembers. “And of course, it filled the room.”
Through the early 2010s, Mozilla engineers and Rust volunteers worldwide gradually honed Rust’s core—the way it is designed to manage memory. They created an “ownership” system so that a piece of data can be referred to by only one variable; this greatly reduces the chances of memory problems. Rust’s compiler—which takes the lines of code you write and turns them into the software that runs on a computer—would rigorously enforce the ownership rules. If a coder violated the rules, the compiler would refuse to compile the code and turn it into a runnable program.
Many of the tricks Rust employed weren’t new ideas: “They’re mostly decades-old research,” says Manish Goregaokar, who runs Rust’s developer-tools team and worked for Mozilla in those early years. But the Rust engineers were adept at finding these well-honed concepts and turning them into practical, usable features.
As the team improved the memory-management system, Rust had increasingly little need for its own garbage collector—and by 2013, the team had removed it. Programs written in Rust would now run even faster: no periodic halts while the computer performed cleanup. There are, Hoare points out, some software engineers who would argue that Rust still possesses elements that are a bit like garbage collection—its “reference counting” system, part of how its memory-ownership mechanics work. But either way, Rust’s performance had become remarkably efficient. It dove closer to the metal, down to where C and C++ were—yet it was memory safe.
Removing garbage collection “led to a leaner and meaner language,” says Steve Klabnik, a coder who got involved with Rust in 2012 and wrote documentation for it for the next 10 years.
Along the way, the Rust community was also building a culture that was known for being unusually friendly and open to newcomers. “No one ever calls you a noob,” says Nell Shamrell-Harrington, a principal engineer at Microsoft who at the time worked on Rust at Mozilla. “No question is considered a stupid question.”
Part of this, she says, is that Hoare had very early on posted a “code of conduct,” prohibiting harassment, that anyone contributing to Rust was expected to adhere to. The community embraced it, and that, longtime Rust community members say, drew queer and trans coders to get involved in Rust in higher proportions than you’d find with other languages. Even the error messages that the compiler creates when the coder makes a mistake are unusually solicitous; they describe the error, and also politely suggest how to fix it.
......
http://surl.li/ewows
No comments:
Post a Comment