What are Guild Talents?

This was recently changed from being a talent tree that was chosen by the guild leader to guild 'perks' (explained in a blue post here (Thanks to Wikwocket)) meaning that each guild will have access to all of them, provided they attain the required level.

As far as the actual perks themselves go...

Money Makers / Savers

• Cash Flow (Rank 2) Each time you loot money from an enemy, an extra 10% money is generated and deposited directly into your guild bank.
• Reinforce (Rank 2) Items take 10% less durability loss when you die.
• Bartering (Rank 2) Reduces the price of items from all vendors by 10%.

Inventory Management

• Mobile Banking Summons your guild bank; 1-hour cooldown.
• Guild Mail In-game mail sent between guild members now arrives instantly.

Travel Benifits

• Hasty Hearth Reduces the cooldown on your Hearthstone by 15 minutes.
• The Quick and the Dead Increases health and mana gained when resurrected by a guild member by 50% and increases movement speed while dead by 100%. Does not function in combat or while in a Battleground or Arena.
• Have Group, Will Travel Summons all raid or party members to the caster's current location. 2-hour cooldown.
• Mount Up Increases speed while mounted by 5%. Not active in Battlegrounds or Arenas.

Tradeskills

• Working Overtime Increases the chance to gain a skill increase on trade skills by 10%.
• Bountiful Bags Increases the quantity of materials gained from mining, skinning, herbalism and disenchanting by 15%.
• Happy Hour Increases the number of flasks gained from using a flask cauldron by 100%.

Misc

• Chug-A-Lug (Rank 2) The duration of buffs from all guild cauldrons and feasts is increased by 100%.
• Mass Resurrection Brings all dead party and raid members back to life with 35 health and 35 mana. Cannot be cast when in combat. No cooldown.
• Mr. Popularity (Rank 2) Reputation gained from killing monsters and completing quests increased by 10%.
• Fast Track (Rank 2) Experience gained from killing monsters and completing quests increased by 10%.
• Everyone's a Hero (Rank 2) Increases heroism points gained by 10%.
• Honorable Mention (Rank 2) Increases honor points gained by 10%.

You can read about how WoW.com rates some of these in the Officer's Quarters posts, Part 1 and 2. There are also perks listed on MMO-Champion

Intro to Guild Leveling

Cataclysm is introducing a guild leveling system, where guilds gain experience as their members complete things (such as winning rated BGs, killing bosses, completing quests and unlocking achievements).

Guild Talents/Perks

It had been planned that guilds would get talent points to spend in talents as they leveled, so they could choose between getting boosts to small boosts in economic, PvE or PvP related areas. The talent points have now been removed and replaced with perks that every guild will automatically get as they level.

Other Guild Bonuses

Players will gain guild reputation as they do things in-game. Gaining reputation for the player gives access to things like tabards, heirlooms and fancy mounts as long as the guild is a high enough level to unlocked them. Changing guilds will reset reputation but leaving and rejoining the same guild should have a limited impact (assuming it is done relatively quickly)

Great question! Personally, I think the benefits of being in a guild is great, and those guild-hoppers annoy me.