The Big Shift

As Americans struggle to make sense of a series of uncomfortable economic changes and disturbing political developments, a worrying picture emerges: of ineffective politicians, frequent scandals, racial backsliding, polarized and irresponsible news media, populists spouting quack economic remedies, growing suspicion of elites and experts, frightening outbreaks of violence, major job losses, high-profile terrorist attacks, anti-immigrant agitation, declining social mobility, giant corporations dominating the economy, rising inequality, and the appearance of a new class of super-empowered billionaires in finance and technology-heavy industries.

That, of course, is a description of American life in the 35 years after the Civil War. The years between the assassination of President Abraham Lincoln, in 1865, and that of President William McKinley, in 1901, were among the least inspiring in the history of U.S. politics. As Reconstruction proved unsuccessful and a series of devastating depressions and panics roiled the economy, Washington failed miserably to rise to the challenges of the day. 

Not many Americans can name the drab presidents who drifted ineffectually through the corridors of the White House during those years; fewer still know the names of the senators and representatives with whom they worked. Almost no one not professionally engaged in the study of U.S. foreign policy can remember a diplomatic accomplishment between the purchase of Alaska and the construction of the Panama Canal. When the politicians of those days are dimly remembered, it is more often for scandal (“Ma, Ma, where’s my Pa?” went the campaign chant referring to President Grover Cleveland’s illegitimate child) than for any substantive accomplishment. 

But if these were disappointing years in the annals of American governance, they were years of extraordinary importance in American history. This was the period in which the United States became the largest and most advanced economy in the world. As transcontinental railroads created a national market and massive industrial development created new industries and new technologies, astonishing inventions poured out steadily from the workshops of Thomas Edison and his imitators and rivals. John D. Rockefeller turned petroleum from a substance of no commercial importance into the foundation of global economic development. The United States’ financial system became as sophisticated and powerful as that of the United Kingdom. 

In hindsight, it was a period in which the United States failed its way to success as the consequences of the Industrial Revolution made themselves felt. The Industrial Revolution began, of course, well before the Civil War, but its full effects were felt only later, as the United States overtook the United Kingdom as the greatest manufacturing power in the world. The rapid technological, social, and economic changes that the Industrial Revolution brought overwhelmed the institutions that had guided the United States since the American Revolution. It was not just the South that found its old political structures and ideas irrelevant in the wake of the war; in the North, too, the political ideals and the governing institutions of the antebellum world no longer sufficed. 

The United States is passing through something similar today. The information revolution is disrupting the country’s social and economic order as profoundly as the Industrial Revolution did. The ideologies and policies that fit American society a generation ago are becoming steadily less applicable to the problems it faces today. The United States’ political parties and most of its political leaders lack the vision and ideas that could solve its most urgent problems. Intellectual and policy elites, for the most part, are too wedded to paradigms that no longer work, but the populists who seek to replace them don’t have real answers, either. It is, in many ways, a stressful and anxious time to be alive. And that anxiety has prompted a pervasive sense of despair about American democracy—a fear that it has reached a point of dysfunction and decay from which it will never recover. 

The effects of rapid change are often unwelcome, but the process of transformation is one of growth and development, not of decline and fall. Indeed, the ability to cope with change remains one of the United States’ greatest sources of strength. In the nineteenth century, people often compared the United States unfavorably with the orderly Prussian-led German empire. Today, the contrast often drawn is with China’s efficient modernization. Yet there is resilience and flexibility in the creative disorder of a free society. There are reasons to believe that, once again, the United States can find a path to an open and humane society that capitalizes on the riches that the new economy will produce. 

LIFE COMES AT YOU FAST

Transitions are painful. In the post–Civil War years, the failure of politics had grave consequences for American life. These were years of mass urbanization in the United States, and government at all levels failed to address the resulting problems. Poor housing, dangerously bad food quality, rampant pollution, high crime, abysmal public health services, inadequate schooling—all blighted urban life in the United States. 

Farm policy was also a disaster. The federal government invited pioneers to settle on increasingly marginal lands west of the 100th meridian; many lost everything they had. The use of machinery and artificial fertilizers raised agricultural productivity, but small family farms were poorly placed to compete. Neither the establishment of land-grant colleges to promote scientific farming, nor the distribution of free land after the passage of the Homestead Act, nor the subsidizing of railroads could change the economic forces that were undermining the security of what for centuries had been the foundation of American society: the family farm. The Industrial Revolution was, among other things, a revolution in how people earned their livings. In 1850, 64 percent of the American population earned its living through farming. By 1900, that figure had fallen to 38 percent, and today, it has hit two percent. 

The Industrial Revolution also led to a decline in social mobility. Before the Civil War, the line between employee and employer was more permeable than it later became. Young people without capital naturally went to work as artisans in workshops, but many of them would soon start their own businesses. As the small workshop was replaced with larger factories, that was no longer possible. Horatio Alger wrote novels about plucky shoeshine boys who rose out of poverty through hard work and good character, but increasingly, society was divided between workers and owners. 

There are reasons to believe that, once again, the United States can find a path to an open and humane society that capitalizes on the riches that the new economy will produce.

As those dividing lines became harder to cross, the classes grew even further apart. There had been rich and poor Americans in 1800—and nearly one million Americans were slaves—but overall, there was much less poverty in the United States then than in most of the rest of the world. After the Civil War, this changed. A class of superrich entrepreneurs and factory owners emerged, crossing the Atlantic to scour Europe for art treasures. Back home, industrial workers labored for subsistence wages, often working 12 or 14 hours a day, seven days a week, in noisy and dangerous factories. 

Financial markets were so poorly regulated that panics and crashes periodically erupted with astonishing ferocity, destroying once flourishing businesses, driving once prosperous families from their homes into the streets, obliterating life savings, and swelling the ranks of the jobless at a time when no social safety net existed to mitigate the horrors of unemployment. 

By the end of the nineteenth century, many Americans were haunted by dystopian visions of a future spinning out of control. The decline of the family farm and the rise of large cities filled with masses of immigrants led many to predict the end of democracy in the United States. Socialists and anarchists agitated for revolutionary change; conservatives feared for the future and saw American values and culture being swamped by immigrants and unfamiliar ideas. 

Yet beginning in the early twentieth century, the United States emerged from the crisis of industrialization to build a new kind of economy that ultimately brought prosperity and freedom to the overwhelming majority of the population. The postbellum generation was witnessing not, as many feared at the time, the death throes of the American experiment but the struggle of a butterfly bursting from its cocoon.

NO PAIN, NO GAIN

The adjustment came in three stages. In the first, from 1865 through 1901, Americans struggled to grapple with the forces reshaping their society. Often, the government was too weak or too poorly organized to undertake the complex tasks that the times demanded. The new ideas that well-intentioned people brought into the arena often fell short of what was required. The bimetal monetary standard of the politician William Jennings Bryan and the “single tax” of the economist Henry George could not solve the problems of the day—but neither could the prophets of orthodoxy solve the problems of agricultural decline, racial inequality, and urban poverty. 

Still, Americans were learning from their failures and deepening their understanding of the new conditions. Economists developed better statistics and sharpened their analysis of such problems as the business cycle and the instabilities of the banking system. Civil service reform improved the quality of government personnel. Social activists and private philanthropists experimented with new methods and new ideas. Theologians rethought the relationship of social problems to the Gospel. New and more forward-looking coalitions began to take shape in American politics. 

Liberal democracy in a world of mature industrial economies turned out not to be the end of history.

The postbellum generation did not solve the new problems that sprang up to confront them, but they laid the foundations for future success. An intellectual, social, and political structure gradually came together that would support a more successful set of policies in what became known as the Progressive era, thus beginning the second stage of adjustment. The Federal Reserve System, regulatory agencies such as the Food and Drug Administration, and reforms such as Prohibition, women’s suffrage, the introduction of the income tax, and the popular election of U.S. senators testified to the growing confidence of a new generation better equipped to deal with industrialization. 

Yet for all these successes, neither the Progressive era nor the more radical and far-reaching New Deal that followed it solved the problems of industrial society. It would take World War II to launch the third and final stage of adjustment. The rapid development of the wartime economy and the large-scale planning necessary to win the war provided Americans with a template for organizing their society more comprehensively than ever before. Only then could the United States harness the full potential of its industrial productivity and create the stable and prosperous society that appeared to have fixed the most basic problems of economic and social life in the modern world. 

In hindsight, the pattern seems clear: the post–Civil War years transformed the United States into the world’s leading industrial economy, and in subsequent decades, Americans would learn how to use the enormous wealth that industrialization created to address the problems that it also brought. By the end of World War II, a rural nation of mostly prosperous small farmers had become an urban and suburban nation of mostly prosperous blue- and white-collar workers. Children went to school, not to factories or mines. The financial upheavals of earlier eras had been largely tamed. The business cycle, if not abolished, had been moderated so that the depressions that shook the industrializing world became a thing of the past. The social safety net protected employees, the elderly, and the infirm from the vicissitudes of life in a market society. Cities had reliable sources of water, gas, and electricity. By the 1970s, the worst of the environmental damage of the Industrial Revolution was being addressed; gradually, the water and the air were getting cleaner, and the slow work of assessing and repairing past environmental damage was under way. 

The peak years of industrial society, from 1945 to 1990, saw the development in many countries of an unusually stable form of regulated capitalism closely aligned with the state. Regulated monopolies and oligopolies dominated many industries. In the United States, AT&T operated the telephone system as a monopoly, and the oil, automobile, airline, and steel industries, among others, were oligopolies dominated by a few large producers. In other industries, such as banking, a large number of firms operated under regulations that limited competition. These employers offered their employees stable jobs with good benefits; increasing numbers of workers received defined-benefit pension plans in addition to Social Security. Wages and benefits gradually rose in real terms. Educational opportunity widened. On the whole, each generation enjoyed a higher standard of living than its predecessor. 

This transformation was not confined to the United States. Throughout the industrial world, the bitter class conflicts of the early decades of industrialization, in the nineteenth and early twentieth centuries, gradually mellowed. After World War II, both capitalists and workers came to prefer compromise over struggle. Socialist parties became more gradualistic, and market-oriented parties became more socially conscious. 

International life also became more stable among the industrial democracies. The restless ambitions of Germany and Japan no longer disturbed the peace. The rise of the European Union and NATO pointed to a new era of deep peace within Europe and in the Atlantic world more generally. 

Beyond the frontiers of the industrialized West, international tensions persisted, with the Soviet bloc and across the developing world. But given the peaceful track record of industrial democracies, it appeared that human political development followed a predictable path. As agrarian societies industrialized, they passed through a kind of adolescence, but over time, they matured. The passions and errors of youth burned away, and delusional fantasies from fascism to communism lost their appeal. In the end, like most well-brought-up youth, these societies grew to become responsible stakeholders in a prosperous world. They led regular lives. 

Much remained to be done, and life in the United States and the other mature industrial societies of the late twentieth century was far from perfect. But their achievements were impressive enough to inspire political scientists to hail liberal, industrial capitalism as the highest and final form of human society. And the idea that industrial development was leading inexorably to both social peace at home and international peace abroad was supremely comforting to an era struggling to come to grips with nuclear weapons and intercontinental ballistic missiles. 

Providence, however, had other plans. Liberal democracy in a world of mature industrial economies turned out not to be the end of history, after all, as geopolitical and ideological rivals have undermined the foundations of the liberal world order in recent years. As horrified EU officials in Brussels followed election returns in the United Kingdom, Hungary, Poland, and Italy, and as the Trump administration sought to take U.S. policy in new directions, the foundations of the building seemed to crumble, even as the architects of world order sought to place the finishing touches on the cathedral of peace. No longer were liberal democracies the custodians of a fully developed and stable social system. Increasingly, the masses no longer believed that technocrats could find the right answers to economic and social questions by following the established procedures. In many cases, citizens of democracies around the world today find themselves in the uncomfortable situation of the postbellum generation: facing problems whose origins cannot be fully understood, and whose solutions will ultimately require intellectual and political architecture that does not yet exist. 

INFORMATION, PLEASE

“Industrial revolution” and “information revolution”—the phrases are so hackneyed that they have come to conceal the very scale of change they were once meant to highlight. Such processes alter society even more profoundly than political revolutions. From the family to the state, from the corporation to the classroom, from the altar to the throne, from gender to finance, no social institution emerges unchanged. Political parties shatter and are rebuilt along new lines; new political ideologies, some of them extreme, emerge from the chaos to command widespread support. It is this kind of change that swept the world during the Industrial Revolution and is sweeping it now. 

For the United States, this is both a pessimistic diagnosis and an optimistic one. On the one hand, the country could face years, perhaps decades, of upheaval and dysfunction as Americans struggle first to understand and then to master the rules of the emerging information society. On the other hand, the journey on which the country has embarked leads not to decline but to achievement. Just as the mature industrial society of the second half of the twentieth century offered the vast majority of Americans greater prosperity, health, and freedom than anyone has enjoyed since humanity emerged from the prehistoric mists, so can one reasonably hope that the creative forces unleashed by the information revolution will give rise to a new kind of society that will be more conducive to human dignity and freedom than anything ever known. 

The information revolution is likely to be even more disruptive than the Industrial Revolution was, and to make matters worse, it is unfolding in an unstable world awash in nuclear weapons. History cannot reveal everything one needs to know about the challenges ahead. Yet as one thinks about how to manage the dangerous period in which the old ways are no longer adequate but the patterns of the new world are not yet clear, reflecting on the last big shift between two forms of economic and social order can offer valuable insights. 

The place to begin is where these revolutions are first felt: in the world of work. The rapid decline of agriculture in the U.S. work force caused by the Industrial Revolution has its contemporary counterpart in the collapse of employment in both factory and routine clerical work. Together, these two categories of labor accounted for nearly half the jobs in the American economy as recently as 1980; as a result of automation and globalization, these two categories accounted for only 15 percent of American jobs in 2016. 

It appears that these historic shifts are only beginning. The much-feared rise of automation is expected to destroy millions of jobs. Although new ones might appear to take their place, no one knows what these jobs will be—or what kind of education to give young people in preparation.

Meanwhile, the nature of employment has changed for many of the jobs that survive. Lifetime employment was often the norm in the mature industrial economy. In the contemporary world, in which companies can spring up overnight and then disappear almost as quickly, lifetime employment is becoming harder to find. Outside the government sector, defined-benefit pension plans have largely disappeared. Not only must many workers change jobs and even industries over the course of their careers, but millions of workers also move in and out of the “gig economy,” driving passengers for Uber, renting out rooms through Airbnb, selling goods on eBay, and taking on various part-time and temporary assignments. 

TIME CHANGE

It will be a very long time before it becomes clear what a mature information economy might look like. If people in the 1860s and 1870s had been told that only two percent of the population would earn its living in agriculture in the twentieth century, they would not have been able to imagine what jobs the displaced farmers could find. That there would one day be “horseless carriages” and even “flying machines” they could perhaps have imagined, but would they have been able to predict that people would get jobs making stickers saying “Baby on Board” that proud parents could place in the rear windows of those carriages? Or that factories would one day grow up to make “travel-size” suitcases that passengers could place in the overhead baggage compartments of the flying machines? Or that health-care professionals would be tasked with writing letters to certify that certain household pets should be designated as “emotional support animals” so that they could accompany their owners on the flying machines? 

The Industrial Revolution and the scientific revolution that accompanied it made it possible for ordinary people to live lives of affluence and security that would have astounded the court of Louis XIV. Automobiles, radios, vacuum cleaners, televisions, and so on transformed the material existence of the masses as well as of the elites. At the same time, the revolution in health care brought a multitude of diseases under control, extended life spans by decades, and found new ways to dull the pain of surgery and childbirth. 

The information revolution will likely have a similar effect on services as cheap information processing continues to bring sophisticated offerings within the price range of the ordinary consumer. The average person will enjoy the kind of information-based services that today are available only to the very rich. Individualized medicine, first-class legal and financial advice and representation, career and professional consulting will be near universal in an age of smart machines and sophisticated software. 

In its early stages, the information revolution has increased the earnings of many knowledge workers. This is likely to change. It was the guild members and tradesmen who suffered the most from the Industrial Revolution. Spinners, weavers, and ironworkers saw their once prestigious and highly paid jobs lose status and income as machines enabled less skilled workers to match their output. If, as seems likely, better software and more powerful computers continue to develop, many of today’s learned professionals risk being replaced by machines in much the same way. Bureaucrats and managers face similar threats. 

The full consequences of the information revolution will only gradually come into view, and the ideas and institutions suitable to it will emerge as the rising generations learn to use the resources and wealth that an information society creates to address the problems it also brings. It is likely that before the adjustment is finished, every institution—from the state to the family to the corporation—will have changed in fundamental ways. In the meantime, people must learn to live in a world of forces that they do not always understand, much less control. “We do not ride on the railroad; it rides upon us,” wrote Henry David Thoreau in 1854. Today, one might say that we do not so much surf the Web as the Web surfs us.

HOW TO GET THROUGH IT

To the extent that the experience of the last great transition offers some useful lessons, the first would be to remember that however difficult this one may appear, what is happening now is an opportunity, not a disaster. While one can mourn the loss of the stable livelihoods, assembly-line work is not, on the whole, an enriching or fulfilling pursuit. That humanity may soon be able to provide for its material needs without conscripting millions of people into lives of repetitive toil is cause for celebration.

Beyond that lesson, it seems clear that many existing social institutions and policies, however well they used to work, need to be changed. The contemporary approach to schooling was developed in response to the needs of the Industrial Revolution: it provided basic literacy to the whole work force, offered more advanced schooling to a percentage of it, and socialized children into the unique working environment of the industrial age. Whether toiling on the factory floor or in corporate or government bureaucracies, the employees of the industrial era worked in hierarchical organizations that valued order and patience. Increasingly, today’s educational system socializes young people into a world that no longer exists.

It is becoming increasingly clear that information is one element, and perhaps the most important, of state power.

The rise of large and stable employers made them central to social policy. The corporation collected taxes on behalf of the state, but it also served as the point of intervention for many social goals. Society looked to employers to provide pensions, health insurance, and a host of other benefits. If, as seems likely, the great firms of the new economy, such as Facebook and Google, will employ fewer people directly than such companies as AT&T and U.S. Steel did in their heyday, and if many firms will hire and fire workers more rapidly than in the past, then it is time to rethink the close relationship between employers and the state. 

Especially during the transition period, it is likely that many Americans will be self-employed or work for very small enterprises, perhaps on a part-time or freelance basis. Ensuring that these companies, on the one hand, have access to credit and other necessary services and, on the other hand, are not crushed by paperwork and compliance burdens designed for larger enterprises will require many policy changes. Everything from zoning laws (to allow people to run small businesses out of their homes) to rules about employer benefits will need to be reviewed. 

One of the most important ways that the information revolution can radically improve the lives of most Americans involves health care. In the future, the human physician may well be a worse diagnostician than a computer program, much as the folk hero John Henry was defeated by the steam drill, or the chess master Garry Kasparov was beaten by IBM’s supercomputer Deep Blue. The characteristic effect of the information revolution on a particular industry is a radical reduction in costs accompanied by a radical enhancement in quality. In health care, then, the primary focus of reform ought to be a shift away from providing a uniform bureaucratic experience to the entire population and toward encouraging the innovation that could someday give every individual American significantly better health care.

Government should also be transformed. The modern civil service is, above all, a product of the Industrial Revolution; the information revolution makes it possible, and perhaps necessary, for government to become much more responsive and effective. Industrial bureaucracies aimed for uniformity: every person who contacted the bureaucracy would ideally be treated in the same way. This works reasonably well when it comes to processes such as granting driver’s licenses, but it works much less well when it comes to delivering complex services. In the future, a worker who loses a job may receive a voucher for unemployment benefits and retraining, and be able to choose among competing firms that offer the kinds of specialized services that a conventional bureaucracy simply cannot provide.

Finally, it is becoming increasingly clear that information is one element, and perhaps the most important, of state power. The “revolution in military affairs” that defense analysts spoke of in the early 1990s—the idea that in battle, the advantage goes to the military that can command the “infospace”—was merely a foretaste of what is to come. Given the increasing importance of signals intelligence, cyberwar, and “big data,” the power that information will give to states is only going to grow. This raises fundamental questions of sovereignty, security, and, of course, civil liberties that will need to be answered. 

The challenge is immense. The foundations of societies are quaking at home, even as the international order threatens to splinter. In the United States, policymakers and politicians now find themselves accountable to a public that may become defensive and antagonistic under the stress of economic and cultural change. The old answers in the old textbooks don’t seem to work anymore, the new answers haven’t been discovered yet, and those who will someday write the new textbooks are still in primary school. To reflect on the upheavals that accompanied the Industrial Revolution—the most destructive wars and the most unspeakable tyrannies in the history of our species—is to realize just how much peril we face. 

Yet humans are problem-solving animals. We thrive on challenges. Americans, for their part, are the heirs to a system of mixed government and popular power that has allowed them to manage great upheavals in the past. The good news and the bad news are perhaps the same: the American people, in common with others around the world, have the opportunity to reach unimaginable levels of affluence and freedom, but to realize that opportunity, they must overcome some of the hardest challenges humanity has ever known. The treasure in the mountain is priceless, but the dragon who guards it is fierce.