Aside from the intimate avenue of family and relationships, agents can attack in a far more impersonal manner. The crudest method of attack happens via harassment by strangers who are part of the Matrix hive-mind network. When walking down a street, through a shopping mall, around a university campus, or riding the subway, etc… one must realize that at least half of the strangers around are organic portals, and most of the rest are asleep souled humans.
All of them are open to temporary animation by the Matrix for direct interaction with a targeted individual. They may mumble cryptic phrases with synchronistic meanings, yell offensive statements tailored to press one’s emotional buttons, and rarely but occasionally engage in a physical attack. What separates them from ordinary crazies or random acts of harassment is that their actions seem unrelated but are far from random. There is timing, engineering, and synchronicity involved in what they do that singles out a specific target at a time.
They could be OPs looking for a quick gulp of energy from vulnerable targets, but the Matrix ensures that whom they pick “deserves” it. Other examples of hive-mind harassment may include hateful stares by strangers for no apparent reason.
Souled people may consciously do these things as well, but there is selfish reason for what they do, whether boredom, general hatred or anger, or just for fun, rather than direct manual control by a hive-mind. With intuition, one can sense the difference. With agents of the Matrix, there is sign of a single coordinated intelligence behind multiple strangers.
When staring into their eyes, one is looking directly into the eyes of the Matrix and the negative beings that maintain it.