Apparently this plant can cause contact dermatitis in people with sensitive skin.
At first, I had this confused with verbena, but commenters quickly corrected me there. :)
In any case, the rough edges of those leaves can create skin irritation. I found this out while reaching through a this plant to remove a rather difficult weed. Then I looked it up on the internet, and found out that both lantana and verbena are indeed known to cause skin irritation in some people.
Here's a lovely shot of my arm.
The picture is a little blurry, so the irritation does not show up very well, but I was starting to get a really nice rash all over the inside of my forearm from reaching through the plant to get at a very obnoxious weed. It itched like crazy, but most of it went away in a few hours (it helped that I washed the area after the irritation began), and it is nearly all gone now, just over 24 hours later. And the discomfort was nothing like what I had later the same day, when my big toe had a run in with some angry fire ants.
I am now searching for alternative low-growing shrubby ground cover type things to replace it. Preferably something that doesn't need heavy pruning every winter. More on that as I learn what might work, and what won't.
Update: Dermatitis from this plant can last longer than 24 hours, despite oral antihistamines and repeated applications of hydrocortisone cream. The rash is still there. I touched the plant on Monday. It is now Thursday. Grrr. Sleeves are highly recommended.