Alzheimer’s disease (AD) is characterized by a progressive dysfunction of central neurons. Recent experimental evidence indicates that in the cortex, in addition to the silencing of a fraction of neurons, other neurons are hyperactive in amyloid-β (Aβ) plaque-enriched regions. However, it has remained unknown what comes first, neuronal silencing or hyperactivity, and what mechanisms might underlie the primary neuronal dysfunction. Here we examine the activity patterns of hippocampal CA1 neurons in a mouse model of AD in vivo using two-photon Ca2+ imaging. We found that neuronal activity in the plaque-bearing CA1 region of older mice is profoundly altered. There was a marked increase in the fractions of both silent and hyperactive neurons, as previously also found in the cortex. Remarkably, in the hippocampus of young mice, we observed a selective increase in hyperactive neurons already before the formation of plaques, suggesting that soluble species of Aβ may underlie this impairment. Indeed, we found that acute treatment with the γ-secretase inhibitor LY-411575 reduces soluble Aβ levels and rescues the neuronal dysfunction. Furthermore, we demonstrate that direct application of soluble Aβ can induce neuronal hyperactivity in wild-type mice. Thus, our study identifies hippocampal hyperactivity as a very early functional impairment in AD transgenic mice and provides direct evidence that soluble Aβ is crucial for hippocampal hyperactivity.