Dirt bottom of pool

I have an above ground pool with lots of dirt at the bottom that I cannot get rid off. After I vacuum it , it looks clean but after I ran it the dirt come back. I used the catfish vacuum which picks up some of the dirt. Please help.