2nd seizure

I went to work Friday at 2:30 and by 4pm my daycare lady was calling me with news I didn't want to hear. Ryan had his second seizure- febrile. I left work and brought him to the ER. Where we waited for 2 1/2 hours to before a doctor saw him! His temp. was 100.4 by ear at daycare. I gave him some tylenol before we left daycare. At the ER he was 99.2, but 2 hours later he was 100 again. Which isn't very high, but Ryan apparently has a very low tolerance. He still isn't 100% back to himself. The Dr. said he probably has something viral and if he has a seizure again I can just watch him at home- that I didn't have to bring him in. I don't know how I feel about all this. I know this is a common childhood thing, but I feel like it shouldn't be just brushed off like it is no big deal. I am worried about my little boy! I have been just taking him to family practice dr. , but tomorrow I am getting him a peds dr. I need to know that this is just a febrile seizure nothing else. They did no tests at the ER.