Fruits are great carb sources. Vegetables can vary: corn has more carbs than something like carrots or spinach. You will have to start checking nutritional data on your own for the foods you eat.
Nutritiondata.comPotatoes are a good source of carbs, I would also recommend eating the skin of the potato as it is full of fiber. Another great source of healthy carbs are beans. You can mix beans in with your vegetables if you need to.
Always choose fresh fruits and vegetables over their canned counterparts, but if your budget doesn't allow for this, buy cans without syrup or with as little syrup as possible.
As far as grains go, you want to eat whole grains (steel-cut oats are a good example, avoid instant oatmeal). For grains you want to emphasize high-fiber, whole-grain, multi-grain, low glycemic index (GI) sources. These are complex carbohydrates and will give you a better slow-burning energy during your day.
Also check out:
Eating Right: How to Get Started by Chris Salvato
This provides several examples of foods you want to start eating to clean up your diet.