News
November 22, 2023
In today's rapidly evolving tech landscape, the rise of artificial intelligence (AI)
has sparked curiosity and concern
among students and aspiring software engineers. As AI and machine learning continue to advance, many wonder:
will AI
replace software engineers?
To address these concerns and shed light on the complex relationship between AI and
software development, we'll explore
the current applications of AI-assisted software development, the benefits it offers, and the potential
challenges it
poses. Additionally, we'll discuss whether AI is poised to replace software engineers entirely or if their
role is set
to transform. Let's dive into the dynamic world of AI and software engineering!
14 February 2024
GitHub has announced applications are now open for the next cohort of its Accelerator program, which provides funding, mentoring, and other benefits for early-stage open-source projects. There is a particular focus this year on developers building AI solutions. Applications will be accepted on a rolling basis until 5 March 2024. Ten projects will be selected to participate in the 10-week program beginning 22 April 2024. The 2024 GitHub Accelerator cohort focuses on the complex challenges developers face when trying to build sustainable businesses around open source AI innovations. While AI advancements rely heavily on open-source collaboration, monetising such work comes with hurdles. “On top of the time and funding obstacles we’re already familiar with in open-source, the heightened expenses and ethical, security, and legal considerations are daunting,” writes GitHub in its program announcement. “We see these challenges as a risk to global innovation and are hoping to help.” Selected projects will receive $40,000 in non-dilutive funding each, as well as extensive mentoring on business strategies tailored to open-source AI startups. Participants will also get credits for AI services from Microsoft Azure and other GitHub partners. At the conclusion of the program, projects will present their work during a demo day attended by GitHub leaders, potential investors, and community members. To be eligible, projects must have an open-source license, clear governance strategy, focus on AI/ML/AI infrastructure, and a plan for using the provided funding. While any stage idea can apply, GitHub says those with active users and a desire to work full-time on open-source are most likely to get selected. The application is free. Project leaders will need to provide information about their work, team, contact details, and links to their code repositories and websites. You can find out more about the Accelerator and how to apply here. (Image Credit: GitHub)
AUG 3, 2023
A FEW YEARS ago, the discussion forum Hacker News, where engineers collectively decide what other engineers
should read, developed a quirk. A new phrase had entered the coder lexicon, and it seemed to propel links to
the top of the page with such force that to some the rankings might have looked rigged. The
phrase—“local-first software”— had an artisanal, farm-to-table sort of ring, at once familiar and flicking
at
something new. Perhaps some engineers dismissed it as merely a marketing term. But others whittling away
their
workday afternoons seemed to see it as the solution to a problem they had long sensed: The software they
were
writing was broken.
One of the first Hacker News links referred to a white paper published in 2019, coauthored by a computer
scientist then at the University of Cambridge named Martin Kleppmann and a group of open source developers
at
an independent “industrial research lab” called Ink & Switch. Kleppmann and the others were alumni of
successful tech startups that had done what successful tech startups are generally meant to do: be acquired.
They had taken a turn inside their larger buyers and emerged repentant, disappointed with certain aspects of
their industry. There were more software developers than ever, but they weren’t coding better experiences
for
their colleagues or their users. They were coding for the cloud.
The lament was not exactly new. A slogan printed on bumper stickers, T-shirts, and water bottles in Silicon
Valley has long mocked the hometown industry with the statement “There is no cloud. There is just someone
else’s computer.” That “someone else” being a corporation. Come to Sand Hill Road with an idea for a
consumer-facing app, and there are two routes to a check sizable enough to get you written up in TechCrunch:
Either monetize your users’ data for resale or advertising, or charge them a fee for accessing that data.
Whichever cloud-based business model you choose—“Senator, we run ads” or “Pay us or else”—it’s imperative
that
the data run through your own servers.
The local-first white paper (“manifesto” might be the more appropriate term) pointed to a third way. The
beauty of the cloud, for the average user, is that it is accessible from many devices and allows
collaboration
between many people across rooms and continents. The authors proposed to keep all that, but with software
that
was essentially cloudless. The word “local” in the name refers to your personal computer. “First” means your
computer is prioritized over “someone else’s.” If you and I wanted to work on a document together, we would
no
longer have to depend on some Google data center in the Oregon high desert to maintain the master copy.
Instead, we would each have copies stored locally on our devices’ hard drives. I could edit my copy offline,
and you could edit yours, and the two files would reconcile our changes anytime they connect, whether once a
minute or once a week.
To build products like this would require fundamentally different ways of structuring data. Different math.
The result of that effort? Less shitty software. Freed from worrying about backends, servers, and
extortionate
cloud computing fees, startups and indie developers could skip strings-attached VC funding and pursue more
interesting apps. What’s more, they could take advantage of hardware improvements that cloud developers
often
missed out on. When an app is cloud-based, its performance is limited by the speed of its connection to the
central server and how quickly that server can reply. With a local-first app, the user’s device runs all the
code. The better your laptop or smartphone gets, the more the app can do.
To a developer, the opposing trends of accelerating machines and stagnant load times are rather silly.
Offensive, really. You should be offended too, because it means you’ve been missing out on something. The
cloud appears heavenly, until it isn’t. Haven’t you noticed, lately, as belts tighten across Silicon Valley,
that your own personal internet feels less abundant than before? That certain things are getting a little
more
expensive, or a little less convenient? A monthly cost to keep all your photos in storage or back up your
phone. A premium upgrade to allow multiple users to edit the same file. A video game that requires a
subscription and lags just as you go for the win.
The journalist and science fiction author Cory Doctorow uses the term “enshittification” to describe how
platform capitalism squanders useful technology. A new platform, flush with venture capital, is first good
to
its users. Then advertisers come for its audience, and the platform is good to them too. Then, still hungry
for profits, it poisons the well. It begins to interfere with the features that you value until you’re fed
up.
This is “how platforms die,” Doctorow writes. Cold business logic blazes this sorry path, but technological
choices pave it.
Maybe that’s fine. Maybe the process is regenerative, like a wildfire clearing the underbrush. It makes way
for new platforms to be good to us again, at least for a while. But what if something different could take
root? What if changing the innards of software, invisible to most of us, could help nudge technology out of
the shit?
KLEPPMANN AND I are suspended three stories above the parking lot of St. Louis’ City Museum, an old shoe
factory turned architectural playground and salvage yard. It’s closing time, and the guards would like us to
climb down and clear out. Kleppmann, however, is aiming for the highest point of the structure, a
hollowed-out
business jet from the 1960s, accessible through a steeply angled tube made of chain link. He wears a royal
blue sweater that somehow instantly clocks him as European, his frizzled orange-brown hair pulled into a
tight
ponytail. As he slides into the fuselage, I imagine I’m pursuing a fox.
The night at the museum is Kleppman’s favorite part of Strange Loop, which might just be his favorite
developer conference. It’s an event that merges joy and weirdness with practicality—his ideal combination.
Kleppmann is perhaps best known for a textbook called Designing Data-Intensive Applications, which explains
the fundamentals of moving lots of data around vast computer systems. A quirky survival guide for the modern
dev, it has sold more than 200,000 copies—enough to merit celebrity status in this community. Fans stop
Kleppmann by the yawning mouth of a life-size whale sculpture and as he emerges from a five-story slide,
thanking him for helping them land their first software jobs.
The seeds of the local-first manifesto can be found in a little box on page 174 of Kleppmann’s book. It
describes something called a conflict-free replicated data type, or CRDT, which he defines as a “family of
data structures” that allow many people to collaborate on a file and “automatically resolve conflicts in
sensible ways.” In the book, Kleppmann notes that the implementation of CRDT algorithms is “still
young.”
By computing standards, though, CRDTs themselves were old. They were co-developed by a French computer
theorist named Marc Shapiro about two decades ago, when the cloud revolution was still nascent. Shapiro, who
subscribed to many of the ideals of the peer-to-peer movement, was beginning to fear where cloud computing
might lead the web. While the internet protocol itself remained open and decentralized, the stuff being
built
on top of it was moving in a monopolistic direction. The tech companies were growing beautiful gardens to
lure
users in—then building walls to discourage them from leaving.
One area they hadn’t yet totally conquered, however, was online collaboration. Connectivity wasn’t good
enough
at the time. Shapiro and his colleague Nuno Preguiça wondered: Did people have to be online to collaborate
online? Or could they work offline and collaborate peer-to-peer?
Conceptually, it wasn’t so difficult to imagine: Create many replicas of the same file, each of which
automatically snaps to a state identical to its peers, like atoms in quantum entanglement. Whether you edit
your replica first and then receive my changes, or I edit my replica and then receive your changes, the
algorithm produces the same result for us both. In math-speak, that’s the “commutative” property. (In fact,
it’s what the “C” in CRDT initially stood for.)
How should the algorithm go about this? In most cases, the answer is straightforward. If I add one paragraph
and you remove another, the order doesn’t matter. But suppose we each tinker with the same word; you think
it
should be purple and I think it should be mauve. What stops the result from being purmaupleve? Different
CRDTs
solve this with different rules meant to preserve the intent of the various collaborators. They may rely on
timestamps to order the new elements, or perhaps have some way of encoding the relationship of each element
to
the elements around it, preserving some notion of words or sentences. The possibilities are
numerous.
These tricks to preserve order can also make the CRDT horribly inefficient. It’s too much data to keep track
of. So the other task for designing a CRDT is one of editing: deciding the minimum amount of information
that
replicas need to send each other in order to produce a harmonious result, and how to package those changes
efficiently.
Shapiro and Preguiça initially published their CRDT algorithm as a technical report. Shapiro thought about
starting a company focused on collaborative editing. “A few months later, bang, Google Docs comes out,” he
tells me. The new software used an older process for merging changes called operational transformation, or
OT,
and still relied on a central Google server. Shapiro believed he had invented something that was more
theoretically sound—a stable foundation for truly peer-to-peer software. But by the time Kleppmann came upon
his paper years later, few people were using the software.
Kleppmann had grown up in Germany toying with both computers and his viola. After an abandoned flirtation
with
a career in composition (ivory tower notions of “what was good and what was rubbish” did not agree with
him),
he had followed the classic techie career arc: He cofounded a startup (called Rapportive, it integrated data
from social media profiles into email contacts); he moved to the Bay Area (closer to investors and social
media giants); his startup was acquired by a tech juggernaut (LinkedIn). Kleppmann lasted a few years before
leaving to take a research position at Cambridge.
The new job gave Kleppmann what he had long wanted: a return to creativity. He had flexibility to explore
more
unusual approaches to programming, including projects that might not immediately pay off. He explains his
work
with an analogy borrowed from his wife, a high school chemistry teacher. If you think of individual bytes of
data as atoms, then data structures are like molecules. For any new coder, the next step after “hello,
world”
is to learn these arrangements—lists, trees, hashes, and graphs, to name a few broad categories. What
Kleppmann wanted to uncover were stranger atomic arrangements that could enable different types of
applications.
He describes Shapiro’s paper as “an awakening.” In CRDTs, Kleppmann saw the technical basis for a new class
of
software that no one was providing. But the algorithms were mostly useless for professional programmers.
They
were too inefficient and lacked the typical tools that developers actually use to make apps. Kleppmann
realized he would have to make local-first developers’ lives easy, shepherding the idea from a set of
mathematical proofs to production-ready code. He set about coding an open source implementation of CRDTs,
which he called Automerge, that people could freely use to build apps.
I SAW THE fruit of this effort a few years later, shortly after the local-first manifesto broke Hacker News.
I
met Peter van Hardenberg, one of Kleppmann’s coauthors, in a café in San Francisco. He was, like Kleppmann,
rebooting after a long journey through the cloud, first as part of the founding team at Heroku, which helped
other startups get their cloud services going, and then inside its acquirer, Salesforce. He wanted to show
me
an app called Pushpin, envisioned as a digital corkboard.
Van Hardenberg pulled up a blank project on his iPad. I loaded a replica of the same file on my laptop. We
began tinkering, adding images and text boxes to our own files, and then allowed them to merge. Sometimes
this
worked seamlessly; other times the changes stopped loading, or the pixels dragged with dial-up-era latency.
Pushpin felt like a toy, the sort of app that a couple of bright-eyed Stanford undergrads might code in the
common room with visions of a seed round and later shelve in embarrassment.
But van Hardenberg was far from embarrassed. The technical groundwork was being laid, he believed, for
local-first versions of Slack, Discord, Google Docs, Photoshop. Better design apps, calendars, budgets. More
complex programs, too, if they could make Automerge far more efficient. There was the possibility of
private,
end-to-end encryption for all these collaborative apps, since no server would get in the way. There were
technical limits to CRDTs—and plenty of applications that the cloud would serve far better. But to him, the
prototype felt like a revolution. There wasn’t a server between us. Yet it worked. Mostly. We were two peers
communicating, as the first bricklayers of the internet intended.
Van Hardenberg’s vision was somewhat easier to see when we met again in St. Louis. The tech giants were
slipping. Meta’s stock was at a seven-year low. Twitter was in the midst of a hostile Elon Musk takeover.
Kleppmann was spending a few hours each week as a technical adviser to Bluesky, spawned by Twitter as a
decentralized experiment and now suddenly thrust in the spotlight, poised to become its competitor. Its
“federated” design promised to give people the option to leave servers and services that treated them
poorly.
Bluesky wasn’t using CRDTs, which would be much too slow for coordinating the feeds of millions of social
media users, but the goal was similar: a better relationship with “someone else’s computer.” Computing
alternatives were once again in vogue.
Among them, CRDTs. Strange Loop was teeming with local-first presentations—a surprise to Kleppmann and van
Hardenberg, who had until recently kept track of every project through Google Alerts and word of mouth.
CRDTs
were turning up in the wider world too. Developers at The Washington Post had used them to build a tool for
arranging articles on the homepage. People poking around in the code that runs Apple’s Notes app had noticed
CRDTs. Jupyter Notebooks, a popular data science app, restored its collaboration tools using CRDTs after
Google got rid of the cloud service it had previously depended on.
Among the presenters at Strange Loop was a Canadian developer named Brooklyn Zelenka, cofounder of a company
called Fission. When she read the local-first manifesto, she recalls, “I was like, this is a great phrase.
Before that, we had these awkward phrases, like ‘location independence’ or ‘user-owned data.’” Zelenka had
been interested in the ideas of Web3—the moniker adopted by “decentralized” apps that use blockchain
technology and cryptocurrency—but found its culture “aggressive,” which she attributed to the focus on money
“so clearly, all the time.” It was nice to be getting into local-first early. “Everything is low-hanging
fruit
right now,” Zelenka told me.
Hers was a common trajectory. Crypto “brought out all the worst people,” van Hardenberg told me over lunch
at
the conference, but it also spoke to many of the same principles as local-first. In his opinion, it simply
uses the wrong approach, promising users decentralization and independence but tethering them to speculative
financial incentives. It’s also the opposite of offline-first: Cumbersome blockchains, controlled by whoever
hoards the most resources, mediate every interaction. Still, crypto did offer a lesson in how hype can fuel
the creation of new products. Van Hardenberg noted the large numbers of bored and disaffected programmers at
the likes of Meta and Google who jumped ship at the height of the crypto bubble.
Local-first, he thought, could eventually elicit the same excitement, but with software that was actually
good. What it needed, van Hardenberg said, was a big “exit” that would bring “signs of visible wealth” to
some
lucky group of local-first developers and help attract more talent and resources. Growth was also scary. Van
Hardenberg and Kleppmann had so far eschewed venture capital funding for Automerge itself, fearing that it
would force them into any variety of business models that “totally go against the values of local-first,” as
Kleppmann told me. But at some point, they realized, growth would also be necessary. They hoped the software
could stand for itself. “VCs love a replatforming,” van Hardenberg said.
A FEW MONTHS after the conference, “local first” was trending again on Hacker News. One commenter called
CRDTs
the “dragon slayer” sword that would allow local-first apps to compete with the cloud. Another lamented that
every interesting technical post about CRDTs devolved into a “weird political discussion about
decentralization.”
Despite many movements for tech decentralization, the dragon’s hoard of gold has kept growing. One problem
is
the perception that principles come at the expense of convenience. As fulfilling and virtuous as it might be
to hew your own coffee table, it’s also hard. Eventually you get tired, and you buy your next piece of
furniture off Amazon. So it goes for managing your data. “It’s so much easier to be lazy and let Apple or
Google do it for you,” Shapiro told me. When I asked him what it was like to use the modern internet while
adhering to his principles, he said he simply abstains from technology as much as he can. “It’s a terrible
waste of your time,” he told me.
I was curious whether the term “local first” bothered Shapiro at all—whether he took it as an unwelcome
rebrand of his technical creation. I was surprised when he told me loved it. There was magic to the phrase,
he
thought. Maybe the revolution has to be a little sneaky to land a blow: reel in developers with the
technical
possibilities, call it a “movement” to attract the politics-obsessed journalists (hello). Maybe it also
needs
to arrive at the right moment, when the Big Tech platforms seem ready to crumble, revealing the features
lost
and the abuses endured in exchange for convenience.
Kleppmann wasn’t demanding a return to analog or to smash all the cloud servers, which do many useful
things.
The sword was not a slayer but a tool to carve out something better—and even he would say that it still
needs
sharpening. When I asked him whether I could try out a new, CRDT-based text editor he had been working on,
his
usual expression of calm attentiveness transformed briefly into alarm. Of course, theoretically I could run
the prototype that’s posted online, since it’s open source—“but please don’t do that,” he told me. He would
let me know when local-first was ready.
April 22, 2022
Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps Perfecto Tests Progressive Web Apps
