Artificial synapses and neurons are recognized as key elements in building bioinspired, neuromorphic computing systems. However, synaptic and neuronal elements that have compatible material systems with each other with high scalability and biorealistic dynamics are yet to be demonstrated. Here we report a two-terminal memristive synapse that can realize short-term and long-term plasticity in both potentiation and depression processes. The Ag nanoclusters introduced at the interface can move, connect and redistribute in response to applied pulses, where their electrochemical migration and thermodynamic relaxation in dielectrics compete with each other and faithfully emulate the synaptic and neuronal dynamics in biology, which in turn allows the same devices to exhibit various synaptic functions and neuronal spiking in a scalable manner. The evolution dynamics of Ag nanoclusters was verified using high resolution transmission electron microscopy and compositional analyses. Based on the inherent state modulator and timing mechanism offered by such dynamics, the devices were able to naturally implement complex functions including metaplasticity, asynchronous classical conditioning and spike-timing-dependent plasticity without needing intentionally designed overlapping pulses, thus paving the way for the construction of intelligent neuromorphic systems capable of encoding and processing spatiotemporal information.