|
|
|
Multihoming for the Small ISPAuthor's note:Please send comments to freedman@netaxs.com with a subject containing 'Multihoming FAQ Comments'. Please send questions to inet-access@earth.com with a subject containing 'Multihoming Question'. If you are not subscribed to inet-access, send e-mail to inet-access-request@earth.com with a Subject of 'subscribe'. Otherwise you probably won't get a response from me. What is Multihoming?You are multi-homed when you have two "upstream providers". These could be other ISPs that you exchange backup service with (more later); large backbones such as MCI, Sprint, UUNet, or Agis/Net99 (PSI and ANS don't sell to resellers); regional backbone providers; or other local ISPs.Why Multihome?Because any one provider may have huge problems at any time. I won't name names here, and the Best Provider of Today could be the Shit Provider of Tomorrow.So the best way to achieve redundancy is to multi-home (connect up to two different providers). Right now, this FAQ only covers "full multi-homing" - not backup arrangements that can be used in case of emergency, but cases where you are fully operational immediately if either (any) of your lines or providers gets hosed.
What is the Job of Your Provider?This may seem obvious, but there are two things your provider is supposed to do for you: (1) Bring data from the rest of the Internet to your network, and (2) Take data from your network to the rest of the Internet.About #1: Your Provider Bringing Data to YouProviders exchange routes with each other. A route is both a description of a section of IP address space and a "promise to accept data" for that section of IP address space. Unfortunately, you must at least know what: routes and IP address space are to get anything out of this FAQ.So how does someone someplace on the Net send data to you? They send a packet to their provider. If you are using a different provider, then their provider sends that packet to your provder based on a "route announcement" having occurred between those two providers. So outgoing route announcements bring data in to your network. Either your provider "nails" your routes into their internal routing table and external route-announcement table somehow, or you announce your routes to your provider and then those routes go into their internal & external-announcement routing tables. Of course, if you got your IP address space from that provider, they will not be announcing your smaller sub-route to the rest of the Internet at the exchange points. [Insert link to the CIDR FAQ.] About #2: You Sending Data to the InternetOne of your machines generates a packet of data, and it's not destined to somewhere within your network. Where do you send it? To one of your provider. How do you make that decision? There's a route in your "border router"(s).Default route: If you have a default route (also written as the 0.0.0.0 route), then you are just sending all data that you don't know what to do with the one of your providers. Even with functional multi-homing, you may still be doing this. Taking routes from your provider: If you have one provider, you *could* take all of the routes on the Internet (about 32,000 of them on 12/15/95). This WILL fit in a 16mb Cisco 2500 series router. But why would you want to do that if you only have one provider? It gives you the same net effect as if you just defaulted into that provider.
Sample Description 1: What if you are single-homed?Usual Configuration:Your provider statically inserts all routes for your network. This means that if you have a customer with a "legacy Class C" that you can't convince to renumber immediately, you'll have to ask your provider to insert a route for you. You default route into your provider.
int e0
ip add [local_ip_adx] [local_ip_mask]
int s0
ip add [local_t1_adx] 255.255.255.252
ip route 0.0.0.0 s0
note: Usually the mask on a serial is 255.255.255.252.
This is to conserve ip address space.
Slighly More Desirable Configuration:You speak BGP with your provider to announce your routes. This way, you can add routes without having to call their NOC or "routing department" to do it for you. If you repeatedly (or even ever, sometimes) screw it up, though, expect to have them either put in a filter list for your route announcements - or switch you to static routing. (Note: We build specific lists of routes that our BGP customers can announce into us to prevent them from blowin nasty routes into us...) You default route into your provider.
int e0
ip add [local_ip_adx] [local_ip_mask]
int s0
ip add [local_t1_adx] 255.255.255.252
router bgp [your-asn]
network [net1]
network [net2] mask 255.255.254.0
network [net3] mask 255.255.252.0
neighbor [remote_t1_adx] remote-as [provider-as]
ip route 0.0.0.0 s0
ip route [net1] dest1
ip route [net2] 255.255.254.0 dest2
ip route [net3] 255.255.252.0 dest3
note: This way ensures that wild routes won't creep in.
But this will ONLY be enough if you have a single provider.
BGP NOTE:If you only have one provider, you don't have to worry as much. If you have multiple providers, you MUST make sure you don't announce garbage routes into them (or routes for all of the net). Providers are usually set up to trust customer routes above any routes heard externally, so if you announce MCI routes into UUNET, you may find UUNET using your MCI T1 to send all of its local data destined for MCI to MCI!!! So you don't have to worry about this until you're dual-homed. Sample Description 2: What if you are dual-homed?Incoming data (outgoing route announcements)Usual Configuration:You speak BGP to both of your providers to announce your routes. What will work: There is NO reason that you "must" speak BGP to your providers. They can both statically insert your routes, but the assumption is that you want to speak BGP because: (1) You want to be able to take full routes from them to make more intelligent routing decisions, and/or (2) You want to be able to add routes without interfacing person-person with them.
Outgoing data (incoming route announcements and/or default route(s))Method 1: Only defaults: Even weightsEither used unnumbered interfaces to your providers, or use the same local IPs on your "serial ends" - your local IP address(es) for your T1s. This assumes that both providers are on the same router at your end. Just put in defaults to both routers with equal weights. Set 'ip route-cache' on both serial lines.You *can* do this while speaking BGP to one or both providers, but if you are speaking BGP you probably want to take "customer routes" (see Method 3). Of course, if you want to speak BGP for outgoing route announcements, you can just filter all incoming routes and know that you can multi-home using this method on even a 1mb Cisco 2501. Method 2: Only defaults: One is a backupHave two default routes - one to each provider, but have one set as a lower weight so that it only takes effect if your link to your primary provider goes down [note: on Ciscos, routes (including default routes) "go away" when the interface that they're associated with is down].This might work when one provider is not as good - and when you don't need the bandwidth of the 2nd connection.
Method 3: Take "customer routes" from each providerSpeak BGP to both providers. Take all of provider X's customer routes from provider X and the same for provider Y. Then, either default into both with equal weights or default into one with a backup default into the other. This will definitely work in a 16mb 2501.
Method 4: Take "full routes" from each providerSpeak BGP to both providers. Take all of each provider's routes for everything. This *might* fit in a 16mb 2501, but probably would not. A 32mb 4000 or 4500 would be fine, as would a Morningstar/gated or PC/gated solution. With this configuration, you *can* run "defaultless" - without a default route, since you should have a fairly specific route for every active site on the net. But there's no need to do this, since if one or both providers go screwy and are not feeding you full routes, you lose connectivity by being defaultless. And having default routes in while taking full routes doesn't hurt anything.
Method 5: The creative route-balancing methodThis is halfway between Method 3 and Method 4. Speak BGP to both providers. Take "customer routes" from each provider. Then, split up the rest of the Internet by their key transit provider's AS:These numbers might be wrong for you! You'll have to look at your routing table (at the AS-paths for ftp.uu.net, ftp.psi.com, ftp.sprintlink.net, etc...) to figure it out. Then, just decide to hear MCI routes through provider A and Sprint routes through provider B. Or whatever mix you want. Balance whose routes you hear through which provider until your average utilization looks balanced. Add default routes to both (equal weighted) or to one as primary and one as secondary to suit.
To be added:
terry@spcvxb.spc.edu Terry Kennedy, Operations Mgr. at St. Peter's College, US In article <1995Dec18.164944.1@hujicc>, yehavi@vms.huji.ac.il (Yehavi Bourvine (58-4279)) writes: An ISP is going soon to be dual-homed to two ISPs which are directly connected to the NAPs. THe question is which router to use: 4,500 with 32MB or 7010 with 64MB. My main concern is the routing tables (full BGP tables) and the amount of memory they use. WIll I have enough free memory on the 4,500 for the forseen future size of the tables, or shall we take a 7010 with 64MB to be on the safe side? 7,000 and 7,500 are out of question due to the price. There are conflicting answers to this question. I posted some numbers a while back from the CIX router, and people told me "the CIX router filters many routes and you'll have problems with full routing". Well, I went ahead and did it anyway. I'm using a 4500M with 32MB main memory and 16MB I/O memory. It is dual-connected to Sprint and Alternet and maintains full routing. Here's the output from sho ip bgp sum and sho mem: router>sh ip bgp sum NeighborV AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State xxx.xxx.xxx.xxx 4 701 432288 20337 2436532 0 0 2d13 xxx.xxx.xxx.xxx 4 1239 411988 20314 2436532 0 0 1w6d xxx.xxx.xxx.xxx 4 xxxx 19984 19978 2436594 0 0 3d04 xxx.xxx.xxx.xxx 4 xxxx 20222 19979 2436532 0 0 4d23 xxx.xxx.xxx.xxx 4 xxxx 19512 19881 2436532 0 0 2d17 router>sh mem Head FreeList Total(b) Used(b) Free(b) Largest(b) Processor 60508B40 604ACA78 28275904 15935728 12340176 11647024 I/O 40000000 604AD78C 16777216 1842080 14935136 14877616 Terry Kennedy Operations Manager, Academic Computing |
||||||||||||||||
|
With any suggestions or questions please feel free to contact us | ||||||||||||||||||