CHAIR: Good morning. We're solving the last little bits of this technical screen trouble and then we'll start.
Okay, good morning. Welcome back to the EIX session. I hope you had a good evening last night and are bright and happy to be here this morning. We will be starting with the co‑chair election this morning. After that we have two updates again. And then a presentation about faster recovery and internet exchanges and then we'll go to the secure information exchange byismism
And last but not list we'll talk about route servers.
So we'll start off with the elections. We're using the single transferable vote system. He knows exactly how that all works. We have three candidates. No new candidates stood before five yesterday. So we have Andy, Wolfgang tremendousful and ex‑co‑. And it's up to you to select your preferred candidates and perhaps second and third candidates if you want. You can vote for either one person or you can vote for all three in order of preference. And the system works this way that the one with the most preference throughout the people that are here plus the ones who voted by e‑mail, we have about 30 e‑mail to include and one they're done counting all the votes they'll let me know and I'll announce the winner. The voting sheets are being distributed. Everyone who is in the room can vote so you're welcome to vote everyone. Please help them distribute the voting sheets and cast your vote. While that is commencing I would like to go to the second part of the agenda which is the BCIX update. It's very quiet this morning. I wonder how that is, why that is.
SPEAKER: There is not enough voting sheets. Right. There's two ways to that. It's basically a simple one, two, three, so if you have a sheet of paper like a general white sheet of paper or someone can run and make copys. Maybe someone could run and get a number of white sheets. Still a few. Okay. So I suggest that someone, and I need a volunteer here, runs to the printer up stairs in the terminal room and gets a couple of white sheets. Wolfegang, thanks a lot. Both of you? Thanks.
BCIX update, tell me is it BCIX or B kicks
SPEAKER: I would like to give the ‑‑ /EUB ma the first introduction about the BCIX at the RIPE meeting. I work for BCIX since last year. Okay. First they were founded as an association in 2002. BCIX runs the Berlin internet exchange and we have current three such locations in about Berlin. We offer peering ports and the usual port speeds and currently we have about 3 gig of traffic. We're an euro‑IX member as well and the executive board consists of five people. This is mixed ‑‑ we have two technical people and quite a lot of finance and Helga for the communication. We have 43 members as of today. Members are quite different like small companies from Berlin, small ISPs, but we have big carriers as well as you can see.
We had a growth curve of about nine new members over the last 12 month. A little bit of loss.
And today we have 40 active peering ports. As you can see here ten gig ports, this was coming very late and that's because BCIX was a free exchange until last August, that means you only have to pay the association fee and peering ports were fee so we should not money to pay peering port so we introduced fees last August and we introduced new switches with ten gig ports.
We have three locations, one is Versatel, at that time they were owned. The second location is carrier‑COLO and the newest one is quite big, 181000 square metre space, have the possibility high power works there. We're proud of this. Technology, we have 4,000, 8,000 and a newer switch.
Between the locations we have darkfibres so we can scale the bandwidth and open BGPD in January.
Okay, here a map of Berlin, you can see the location. We have E shelter on the left and Versatel, which is near to the level three gateway, which is next door to this location.
Okay, route server, open BGPD, I think we'll speak about this later. We run this without problems since January and we have a couple of peers on it as you can see. We have communities but no filtering active on that route server.
Okay. We look for more peers, of course, as other exchanges also, and specially for CDN and content providers because we have a lot of excess networkers on the exchange and need the content to get more traffic.
Okay, if you want to join us, we have port fees, of course. This is 300 euro for gig port and 1200 euro for gig port and 600 euro membership association fee. Okay. That's it. Any questions?
CHAIR: No questions? Thanks a lot.
(Applause.)
Next up is Maurice if he's here. Yes. Some peers ‑‑ I forgot about that, I switched around the agenda from yesterday.
Maybe you could switch it around and do the other presentation first? Yeah, okay.
So in that case, you're up afterall. Lots of technical stuff today. Sorry guys. There you go.
SPEAKER: Good morning, my name is ZBYNEK, and I have an idea about a new function of the equipment of an internet exchange. Still nobody told me it's ‑‑ I would like to present it here.
As we are a user about ten internet exchange point in Europe, we have discovered one of the weakness of the shared medium in the internet exchanges is when a link to one ISP fails, all other ISPs knows about it after one or two minutes when the BGP session becomes timeed out. So how it's possible to solve this problem? It is possible by tuning of timeers of the BGP. What is the way to Hell or by BGP which must be implemented by all ISPs, so it's also hard because not every routeer supports it, not every ISP wants to support it, et cetera, et cetera.
So? My proposal, how to solve this problem is to implement a function to the internet switch in the internet exchange point to the add switch in the internet exchange point. When the switch monitors all BGP sessions goes through each customer access port and when the port goes down, when the port is broken, it takes an IP address of the broken routeer and sends a packet into all BGP sessions in the other end of all BGP sessions going through the affected port. I think it would be ‑‑ it could be understandable for you from the picture.
This is the ‑‑ very easy algorithm how it's possible, how it's possible to implement it into the internet switch, specially to asick or APG circuits in the line cart of the switch. This is the algorithm of learning of BGP sessions. It's easy, the only thing is the switch monitors, monitors all traffic if it's some BGP traffic and if it is BGP traffic, it handles it as describes on the picture.
The same algorithm is for the reset of sessions when the port wheel is down. So if the port goes down, the switch sends RST to all systems going through the affected port. It's really easy only the handling of TCP, ‑‑ of TCP segment number is necessary to handle for this solution.
So what is the advantage? The time between problem detection and reaction could be rapidly decreased from minutes to tens or hundreds of milliseconds.
So it's seemless for users when the problem appears, not as now when you are waiting and you see there is an outage.
There is also known problems. The first problem is somebody must implement this function into their boxes and the second problem, I know about is MD 5 protection. It could be solved by a database of FD pass words in the switch. It's not a good way maybe. It could be. I don't know. It's partly possible to solve it with using TTL security instead of MD5 pass word protecting.
So thank you. Some questions?
QUESTIONER: Steve Kent, BBN technologies.
SPEAKER: Yes.
QUESTIONER: In addition to TTCMD five there's ongoing work i4n the ITF for successor security mechanisms for TTC specifically for protecting BGP sessions and those are automatic which make it completely infeasible to have any passwords in the switch in addition to the fact that you're looking above several layers where the switch should look in the first place.
SPEAKER: Okay, thanks.
QUESTIONER: Stephen backer from AMS‑IX. The idea itself might be intrigueing but I don't how see how it's a going to work on a layer 2 network which involves multiple switches which will need to know all those peering details and how to do it on multiple 10 gig E lined rate. You have to look at every single packet to make sure ‑‑ to check whether it's BGP packet and take action on it. I can tell you already the current switches have a hard enough time to switch the traffic that they have to do instead of just inspecting every single packet and doing something with it. I really don't see this working in any feasible way. I'm not sure what problem you're trying to solve. The most problems is not timing out of BGP sessions but the excessive flaping of BGP sessions due to a short link flap, for example, and on those occasions you don't want your BGP sessions to flap, you want them to hold on for a short while. So it's just going to create more turn, I think, in the routing table than is necessary.
SPEAKER: Okay, thank you.
QUESTIONER: I think you won't get any switch runner to implement that actually. I think internet exchanges are a small fish for the big switch providers and every feature we want we usually have to fight for. And I think this is a feature that's really ‑‑ I think it would not need a lot of time and implementing stuff. So I think you won't get the switch runners doing that. You could have a server doing that. I think that connects to ‑‑ some kind of a weird hack or get traps and it would send out spoof messages but it needs some kind of interference with the switch anyway so I don't think it's working.
SPEAKER: Yes but doing it this way could be much much much slower.
QUESTIONER: Yes, of course.
QUESTIONER: Ruedlger Volk, Deutsche Telecom, abusing security leaks to sneak in stuff just get BFG ‑‑ okay, for those parties who do not want to support it long term, well okay, they may have ‑‑ they may have some peerings denyed because of parties want to have it, well, okay, people who are not up to state of the art have some difficulty.
SPEAKER: Okay, it's necessary to be possible to switch this function down on specifyed port, of course, like any other function including CDP or I don't know. (CDP)
QUESTIONER: Google. I would suggest to you to consider something like extension route server function alternate. When route server is part of the same exchange so you can get from switching platform of the information involved link failure and as you have a peering session, you may have some code extension on route server like the message prior to something. You can basically speed up operation of route server and link it to like link flop conditions, as an example.
And question to you: What is specifically bad in tweaking BGP timeers? Do you have any numbers to back that up?
QUESTIONER: I don't say it's bad (speaker) every manipulation of timeers, specifically down and ‑‑ specifically if on the session is a very, very large amount, amount of updates and prefixes. It's a lot of stress for the CPU of the routeer. And that is a lot of providers who are using software like Cisco and I don't know some bloggers. So it could cause flaping only because the Quagga doesn't ‑‑ can't important all prefixes in the time before the session becomes down. For example, it's my experience with it, yes? So I think it's not a good idea to making bigger stress for peering routeers.
QUESTIONER: Thank you.
SPEAKER: Okay. Thank you.
(Applause.)
CHAIR:
QUESTIONER: RIPE NCC. Could we just have all the ballot papers in in the next couple of minutes so we can count them during the session. Raise your hands with the paper if you still have one. One there, one in the middle. One in the back.
Okay, Maurice, you're up next or now actually.
So Maurice is going to talk about some developments in Paris.
SPEAKER: As we just sort this out, it's actually a quick presentation on not exactly an exchange but an idea of a proposal to investigate building an exchange switch fabric in Paris. We just ‑‑ I'm just going to talk without the assistance of slides.
I work for Google but I'm here under a personal title working for a ‑‑ working for a Working Group on a project we've called phoenix which is to look at the market situation in Paris and see if we can help address some of the short falls we see with the exchange environment there.
We'll start with what we see as some of the challenges. Paris has the fortunate position of being very central to European Telecoms industry. France has numerous submarine cables running across the Atlantic into the Middle East, into Africa and all the way into eastern Australia, with landing stations in south earn and northern France. If you look at the environment, the exchange environment within Paris, there seems to be a surprisingly few ‑‑ few large‑scale exchanges that represent the countries ‑‑ or have members from the countries that are connected via those submarine cables. Also there's something of a fragmented environment where I think at the last count there were 11 internet exchanges within the metro area of Paris. That may have increased since I last counted. They tend to appear and disappear.
In discussions with various networks or network operators and content networks, there's a clear view that this fragmentation, this numerous number of exchanges and the complexities of operating in Paris is a detriment to future growth. So we've formed a Working Group of sorts to take a look at this and see if we can formulate some proposals of how to address this.
Early conclusions are that there is a demand, that demand looks like a demand for a robust, well‑supported distributed switch fabric architecture within the Paris metro area with some features maybe not to be present in all existing exchanges, things like the ability for advance reporting features based on MAC addressing or accounting, S flow, the support for platforms or services that go beyond the normal standard exchange of IP traffic.
So, for example, platforms to support the interconnection of VOIP, maybe video conferenceing and other platforms. And we ‑‑ we're starting to think around how we can address this demand. Well, first of all, we think ‑‑ the structure we're thinking is that we should look at consolidating a number of the existing infrastructure, number of the existing exchanges that may be located in one or two facilities and see if we can't build a fabric in which the participants connected to those exchanges can exchange traffic with other participants in different buildings, different facilities.
The second conclusion we have is the formulation of this idea ‑‑ or the most successful of the type of models like this have been under the auspice of a non‑profit organisation and we think this is probably the best way to approach. And with an emphasis from an operations viewpoint and designing for scale and operating a level of technical and business operations that are focused around performance and availability of the infrastructure.
So that's a quick introduction.
We're trying to get a good understand of those demands in a meaningful way backed up with some data. We originated a survey that we're distributeing hopefully starting today, just asks a few questions around ‑‑ of those ISP facilities and CDBs and general operators who are present or intend to be present in Paris. What their requirements and demands are, the challenges they see. We're going to collect data over the next three weeks to see if we can't have ‑‑ make some sense of it and refine this proposal into this more detail. At that point we'll formulate a feasibility study and if the feasibility study doesn't say no, it's a ridiculous idea drop it immediately, then we'll look at the next steps of forming a commercial Working Group and a technical Working Group to take a look at the proposal in more detail.
That's where we are with the PHEON‑IX idea. If you could go into the website, which the address is here if you have strong opinions or comments, please leave them and we'll be grateful to read through them.
There's also ‑‑ before I open the floor to any questions, we'd also be interested in interesting a discussion with ISPs or facilities that have an interest in Paris. If they're outside of the format of the survey, we want to approach some of the Working Group to have an off‑line discussion. Please approach myself or Christian, we're all there or probably at the bar for the next day or so.
Yes, that's it. Any questions?
QUESTIONER: High. An observation rather than a comment or question. You might want to have a look at your survey and remove the German and perhaps put some French in instead.
SPEAKER: I didn't catch that.
QUESTIONER: Neither did I. Some of the words in your survey are in German, instead of others I see the German word. So your survey is partly in German.
SPEAKER: Well it's taken on a level of artificial intelligence and I'll be patenting the technology and will be available for licence. Thank you. I'll take a look at that. Any other indication questions or indications of my next business model?
CHAIR: No. We'll have a look at that survey, I guess, because that's interesting.
We have a next presentation about the secure information exchange from the ISC (ISC)
Is everything up loaded? No election result yet? Still being counted.
After that, we'll have the best common practices for IXPs and we'll close off with the route server discussion.
With
SPEAKER: Okay. Good morning. I'm Keith Mitchell. Little while since I spoke to anyone, living over in DNS land past few years. Nice to be back. I'm going to talk about a different exchange, working on SIE, Security Information Exchange. I'm sure you're familiar with the concept of exchanges as trusted intermediarys fending off the come pet terrors trying to get them work together. Internet exchanges have been around. There's other kind of exchanges for trading and also interconnect, not just the switching that we often do. What's the idea of a Security Information Exchange? As you all know internet security is big business these days but there are some limitations in the way that people deal with it. The idea of the Security Information Exchange is we can do exchange of security sensitive data in real time and also in a centralized multi‑lateral basis which is something that doesn't happen at the moment and use it as a way of fighting back at the bad guys. This is how security information is shared on the internet at the moment. Lots of different players, players have their own communities, various bilateral options but not a clearing house for this kind of information.
Here's something exchange operators are a little more familiar with, sharing information between the different parties. Everybody contributes. Everybody potentially consumes and analyzes the data coming in, the data shared. Obviously this is sensitive data so it's important to of a policy framework and a legal framework.
So here's the kind of thing we have at the moment, a bunch of sensors out there, passive DNS sensors, so gathering DNS query data but that's scaling up to more than just DNS stuff now. We have some relay machines, more about that in a minute. We have a switch, which is a layer switch 2 fabic but it's used a little different and we have a bunch of reserve doing analysis and summarizing the results of that back other to the exchange. This is in the San Francisco bay area. We're looking to expand this to other locations. We're in the process of building another exchange on the east coast of the US, probably DC and New York. Again another major data center where all the players are. And again there will be sensors feeding that. It will be possible to summarize data between the two different locations. We don't want to do full strength real time feeds because very large volume but some of the data potentially shareable between the other locations.
And then it also makes sense to potentially gather data from other parts of the data internationally and local relays, again to summarize that. And one of the reasons for having local relays is that ‑‑ depending on privacy legislation, limitations, you may not want to move the data between countries. Some of the data sharing you may want to keep within a particular country or EU rather than back halling between international boundaries and potentially causing issues. It's also possible to do analysis on the data, to make sure that personal identifiable information is not being shipped around between the different nodes, which is also important.
And then if you get enough capacity in a particular location, then potentially make that a full node. We're talking to various exchange operators at the moment and we'd be interested in talking to more and setting up locations close to existing peering exchanges.
So getting back to the model, there's a lot of data out there used in the abuse community, spam, DNS black list, passive DNS, log files, anti‑virus, anti‑fishing activities, abuse reports, honey pots, all kinds of things. We've put together some tools, Ncap is a tool you might find interesting, it's designed to parse DNS messages to quite a high level. Various plug ins which can be used for filtering level for broadcasting. We have generalized broad /K‑P. It takes a lot of hardware. One of the decisions we made is we don't store data. This is all in real time. That places interesting demands on the switching hardware, on the server hardware, it means quite important architecture decisions. Hard discs can't cut it. We need lots of ram, we need flash discs and we've been deploying a lot of that.
So NCAP is a derivative of something that was DNS cap. It does some things that are very useful, if you're looking at passive DNS data, it can defrag the queries, gets rid of link layer information as a normalized format, uses accurate time stamps and other features so it copes better, has modules and the deduplication is very important as well.
If you add together all the passive DNS features we have, which are queries from up stream of catching resolving, that's a very large volume of data. So these go straight on to the exchange but there are analysis machines ‑‑ some researchers want to look at all the DNS messages and others are interested in other things, we do deduplication. There are various filters that we can do, for example, so that you see changes between DNS queries for people researching Fast Flux used for controlling botnets. But yeah, we're seeing 201000 packets per second just from passive DNS traffic. That's a lot. If you deduplicate it you get it down to 31000 packet which is easy. If you start running filters on that, so you're only looking at changes, then you have the ability to do Fast Flux almost human eyeball speed, try and figure out who's using Fast Flux DNS to run their botnet.
A little bit about NMSG, where's the PCap, DNS cap? Format was basically a way of rebroadcasting DNS queries. This NMSG is for a whole bunch of different types of structure but still using a broadcast exchange but these can be things like spam headers, used as a generalized protocol buffer format which I won't claim to understand but this is something Google has come up with and seems to be a useful tool for generalized handling of data. Something you don't see often, fragment of code. Don't ask me what it means but it does the necessary stuff.
The way it works is we have a bunch of channels which correspond to V‑LANs, most of these [are] Mac leer broadcast. You wouldn't want to run it over your peering fabric. It needs dedicated switches. We've talks about passive DNS, Fast Flux, various channels for anti‑spam black list, various types of DNS traffic, polar sites TLDs, so you can find the people who are sourceing the PTR look ups that shouldn't be. Various legacy root which are no longer author tiff or used but seek a huge number of queries. Lots more channels. We want more spam. You recall link pairs, net flow data, various sink holes and other things. SIE has proved a grate tool and there's some quite interesting analysis being done.
Potential queries from dark net space, taking chunks of space and looking at what's coming in and also various malware activities as well. One channel example is spam. You can get it from spam traps, supports, and obviously it's not just spam. You can use what you get out of spam analysis to look at other bad activity but the spam is trying to trap people into malware or fishing. How you should represent an e‑mail message.
Mentioned malware analysis. Some people can submit mal wear, provide links to it. There are various tools, these are all in real time, analyzing that. Some people can share data. Other people find this a valuable use of raw data. And there's lots of other things, reputation analysis, DNS scans, the results of going out there and trying to fingerprint all the DNS servers, finding out what version they're running, maybe we can use it for DD OS reporting and a whole bunch of people are going to the same you recall, maybe it's an event or may be it's a malware, maybe we can use it for BGP traffic.
So this is a collaborative effort. If you can help with this we'd much appreciate it. We're always looking for data. Once that data is there, there's the possibility of sharing it. If people want to bring servers, we can host them. And the other thing that's helpful, particularly from ISPs, particularly if you're running large or cursive name servers, is to install sensors and get this data so it can be back halled into SIE and analyzed and it can be used to protect abuse on your network.
How does this work for exchange operators? Another type of value that you can put into your exchange. It's not just about exchanging raw data but about doing analysis on that data. You can, for example, work with your ISP members and help them deal with abuse, which is always a problem, and there's always potential in new tools. You can do outreach to law enforcement. Be more practical about security generally. But obviously the other thing you need to be careful about is privacy laws and policies. This isn't specifically wiretaping but it's looking at data and it's very important to make sure that data is dealt with appropriately.
So that's about it. Our program manager for SIE is in the Jabber room at the moment. If there's any questions I can't field, Eric will be happy to field them, either online now or if you want to contact him afterwards. Happy to take any questions.
No questions?
CHAIR: One question.
QUESTIONER: Hi. Eric from UCLA and CSU. As a researcher, how can I get access to some of this? Specifically the DNS names?
SPEAKER: You need to sign an agreement to make sure the data you're receiveing and analyzing is dealt with appropriately and maintains confidentiality and we can provide you with a server you can plug into and you can tune into one of the channels, passive DNS channel and it's up to you then what analysis you do on that
QUESTIONER: So just provide a box after signing an agreement?
SPEAKER: Pretty much, yeah.
CHAIR: No other questions for Keith?
SPEAKER: Okay, thank you everyone.
(Applause.)
CHAIR: Letter. Then we have a presentation by Wolfe gang, it's about best common practices in IXPs.
SPEAKER: Hello everyone. My name is Wolfgang. I'm talking about IXPBCPs, this is a project that arises from the euro‑IX community. It was originally started by the Italian guy. How was he called? Clare
CHAIR: Simone.
SPEAKER: Yes, he started this and after he left the Italian IXP I took on with that project. It's basically a document for technical practices for internet exchanges. So it goes through set of maintenance of an exchange point infrastructure or ‑‑ you also get a set of recommendations from already established IXPs. So what you basically get there is information how to set up or run an IXP. If you haven't got an ‑‑ if you want to create an IXP, just do it and follow this best current practices document. You will get an IXP that is more or less the recommendation from our community. If you already run an IXP and you want to know how do others do monitoring, how do others have, like, the set up for out of band management and things like that, you will find it in this document. It's not described in whole detail, how do you set up whatever, but it generally tells how most of the IXPs in Europe or members of the Euro‑IX community do that. What it does not address is commercial or political communities.
This is not a commercial for Euro‑IX but they get access to sample configurations from IXPs some contribute some this won't be published because the IXPs decided not to publish their internal /KOB tent for security reasons. They also have the possibility to contribute content. And they have access to more than the general IXP guide Lynn lines. There's a website, there's an up coming route server, or whatever.
So this document will be published on the Euro‑IX website soon. There's still some time for Euro‑IX members to read through it before we publish it. There might be a possibility to add comments for non‑members. Maybe it's just done by e‑mail, if the request isn't too high or a comment section on the web page. But if you have a friend who wants to create an IXP or know how things work, this is a good document to address that.
And I think this is it. Yes. Any questions? No questions.
QUESTIONER:
CHAIR: No, wasn't a question. Okay. Thanks, Wolfe gang.
Before we move to the route server discussion, we have the results of the elections.
SPEAKER: I've counted the election and the number of ballot papers is also double checked by another member of the staff. Quite close run. We had 110 ballot papers from the room. We had 28 e‑mail votes. I've discounted five additional, that's 28 valid. There were five invalid because they came in after 10 o'clock. The last one came in a couple minutes ago. So 138 valid votes all together. In the first round we had 57 votes for Andy Davidson, 36 for Wolfgang and 45 for Remco , so we've taken the votes for Wolfgang and redistributed those and the final result is Andy Davidson 68 votes and Remco 59. So I declare that Andy Davidson is the winner of the election.
CHAIR: Congratulations Andy. Will you come up. I'm just inviting you up here and you can make a...
Andy: Thank you very much.
CHAIR: You'll be up here next time, in Lisbon, I guess.
Okay, very well. So to close off, we still have about 25 minutes. We're doing some discussion on route serving. I would like to close a little bit early, like five minutes, so everybody has enough time to get to the lunch and I have to go to the Working Group chair lunch. So we'll all be happy and early for lunch.
I would like to invite nick who wanted to kick off the route server discussion with a couple of slides.
SPEAKER: Good afternoon everybody.
I want to explain why a lot of internet exchanges use Quagga for a route server. It's not always obvious. There are a lot of BGP stacks out there. The problems start arising when you have a situation where you have multiple members on your exchange and you have ‑‑ do you have a pointer? ‑‑ where you have a requirement to do prefix filtering. Now, not all of the commercial BGP stacks like Cisco and like that like stripping out your ought none mouse number, Quagga and BGP bird will do those. Quagga supports multiple ribs. To explain why this is a good feature I'm going to use this diagram here.
You have here four route server clients, A, B, C and D. A and B are providing connectivity into the exchange for these clients over here, X and Y. So you can immediately see that this path here AYX is longer than BX so you would expect that B would provide connect tiff into the exchange for X. So that's what's going to happen on a single‑rib route server. It will run it's best pass calculation and calculate that the best path is indead BX. But in real life when you're running an internet exchange route server you really want to provide the facility for clients of that route server to implement filtering, outbound filtering from the route server so other people will not necessarily be able to see your prefixes. The usual way of doing this ‑‑ and there are others, and this isn't ‑‑ the problem here isn't specific to using BGP community tags but on a lot of exchanges we use the notation ‑‑ we type prefixes with 0: , whatever the AS number that you don't want to see or receive your routes and then the route server will filter those out. So B here has tagged owl of its prefixes with 0: D because it wasn't want to appear with D for whatever reason, personality conflicts or commercial interests or whatever. So what you'd really like is D would be able to get back to here using this back up path of AYX but it can't, the route server has only a single rib and single best path and that path is BX and therefore D cannot see this autonomous system number over here through the route server.
Quagga gets around this by implementing multiple loc‑RIBs. Very quickly, you have A, B, C, D, are route server clients. Same situation as before, AYX is longer than BX. Now, the route server is different now, you have multiple ribs inside it and in order to implement internal filtering within the route server Quagga implemented export and import route map filters, they're a little bit like in and out except that they work internally.
Okay, all of those prefixes are going to go into all of these lock‑ribs here, prefixes from A are going to be into ABC and prefixes from B are going into BC and D.
So RIB D is only going to see YX, not BX because BX specified it doesn't want the route server to transfit its prefixes to D.
So RIB C will select the best path to being BX here and RIB D will see the best path as AYX.
So that's why you kind of really need multiple Loc‑RIBs. Quagga at the moment is the only easily available BGP implementation with Loc‑RIBs. There are a couple of others not easily attainable.
There are a couple problems. With a regular BGP implementation your resources are linear. If you have a million prefixes, that's going to be ten times as much work as 100,000 prefixes. With multiple lock it's going to go from P to that, which is order N squareed. And that's really horrible. So that's ‑‑ basically if you do on to show that summary, you multiply the number of prefixes received from each client by the total number of clients and that's roughly the order of magnitude of complications for memory and CPU. This is immediately a problem for any BGP implementation. Memory is fine, you can bang in heaps of memory and that will keep you going. But really you need a multi‑threaded BGP implementation. If you do everything on a single core and if you have piles of prefixes and piles of clients like some of the internet exchanges they're going to get very busy. It's fine if you're small they have no problems but the others it hurts.
So that's a brief introduction into why a route server at an Internet Exchange is a little different from the regular BGP stack.
Any questions?
CHAIR: Thanks for that introduction. So that was about a Quagga set up. Now, I've talked to a couple of people here in the room that also run route servers and that have specific set ups, specific choices that they made and they would like to say something about that and why do they tolerate or chose for what they chose for.
Andy, you want to go first? You're building one, you are building two actually?
SPEAKER: I hinted at this yesterday. We want to build route servers because the evidence we've seen around ‑‑ when we discuss this with our colleagues at other exchanges, it's a good way of growing value for people as soon as they connect to the exchange. It's a service that a number of people have individually asked us for and it's ‑‑ it's a good way to grow traffic which helps with marketing. So we've decided we want to build some route servers but didn't want to build another Quagga because there were a bunch of them in London. We're going to ‑‑ we're likely to roll out route servers on open BGP and support the AMS‑IX's to add per peer RIB to that software, and also roll out Bird, which the guys from Prague are doing a lot of work to make good ‑‑ I think we're going to be the first exchange to role that as a route server. We're working really closely with those guys to swap really good feedback. If there's other exchanges in the room making similar decisions, we'd certainly swap information about how successful the new builds are going. That explains our position and why we're not using Quagga.
CHAIR: You mentioned two others. If anyone has a remark or question in between, go up to the mike and state your question and remark.
Now, Andy mentioned AMS‑IX as working on a new set up for BGP.
SPEAKER: We're moving away from ‑‑ we're currently using less Quagga, it's just basically too unstable, it really has scaleing problems and we really see the single‑threading issue. So we're going to ‑‑ we're actually moving to which isn't quite ready yet (BGPD) we're paying developers to put that in and hopefully we're going to see something. We've seen CVS commits on the open repository. Something is happening we're just not really sure what's happening now. It's in development. So...
CHAIR: I think Andre is going to talk about Bird a little bit here.
SPEAKER: Thank you very much. Andre Phillip and as was mentioned, sort of group people, the devil of the Bird, quite an old one but not very well known, and we are currently running two Quaggas as a route server and are not happy. In any case we would like to have solutions so we are working on replacing one of the Quagga by Bird. And many of you probably doesn't know Bird. It's a routing demon which is developed by people who have close relationships with this exchange point. Like, for example, Nick presented those will be hard and the people understand the problematics of those things. So if you need some more features into Bird, just let us know and we can implement some new things into that.
CHAIR: Mike.
SPEAKER: I think one of the things I'd like to get across, we run right now Quagga as I described yesterday and I showed you some stats and number of peers and prefixes we receive on there. We ran Quagga when we first because really that's all there was. There wasn't much in the way of viable alternatives. Route servers have become very vital, I think, to the larger ‑‑ to a lot of the larger exchanges because if you think of an exchange the size of AMS‑IX or links where you have 31000 members ‑‑ if you have an open peering policy trying to maintain that number of peering sessions and that number of peering relationships on a configuring per device per session it's quite a lot of work and you have to ask yourself, if it wasn't the route servers would that peering community and environment be so rich? And probably wouldn't. People would say I'm not going to peer with these hundred people because it's not worth the time, if I can get it via the route server it's low cost and easy pickings.
We're currently in the situation where we have issues with Quagga, hit some of the scaling issues that nick was just talking about, and we're basically just backing several horses in this race because there are lots of alternatives, although there's starting to be a chance of lots of viable alternatives, Bird one of them, open BGPD once it has the ‑‑ (BGPD) but there wasn't that many alternatives, so I think what we're seeing is, what we know is have his /SR‑S a vacuum and it's trying to be filled and there is a vacuum with Quagga with the larger exchanges and we're trying to fill that gap one way or the other. That's just my observation of the state of the nation with route servers at the moment. We're just trying ‑‑ basically done some things to buy us some time with Quagga but that's all we can do until we find a replacement and that's spending a lot of time and effort on that at the moment.
CHAIR: Any other exchange? Harold, go ahead. You're also in an implementation phase or thinking phase.
SPEAKER: As I mentioned yesterday in the introduction, we currently plan to implement a route server as fast as possible but besides the decision which software to take we're intent to take Bird because it's developed by IX people and there's always rhetorical questions, how do we implement the filtering issue and Nick showed it's common to use BGP community text to give the route server information which prefix has to be announced to which peer but this is not compatible with 32‑bit models and we want to build a new route server which is capable with all and I don't want to call 32 numbers a problem but if you build a new route server it should be compatible with everything that could come now. That's our main problem. We want to build something that is able to do all our requirements at the moment.
CHAIR: Anyone? Arnold? Yes.
SPEAKER: Technically, there's not much I can add. Almost everything was said. Currently we at DE‑CIX are still running a fairly old version and quite stable compareed what I seen at other exchanges.
I guess I should also mention that last week at Euro‑IX we talked about it and we decided to form a Working Group on improveing Quagga and we already see substantial improvement in gathering resources both in personal as well as collecting money, and I'm quite sure that next time in the RIPE meeting in Lisbon, we'll be able to provide results on this.
CHAIR: Thanks for mentioning that and I'll save you a lot then.
Okay, I think that concludes. Unless there's anyone else that wants to say something? Okay. That means that we're done for today. I would like to thank you all for being here. I would like to thank Fergus for being remote. He will be here the next time or at least we'll see about that. Andy, congratulations, Wolfgang, thanks for standing, Remco , thanks for standing. Big applause for those guys as well.
(Applause.) See you next time. Roland, also thanks for counting the votes. We're done.
So we'll start off with the elections. We're using the single transferable vote system. He knows exactly how that all works. We have three candidates. No new candidates stood before five yesterday. So we have Andy, Wolfgang tremendousful and ex‑co‑. And it's up to you to select your preferred candidates and perhaps second and third candidates if you want. You can vote for either one person or you can vote for all three in order of preference. And the system works this way that the one with the most preference throughout the people that are here plus the ones who voted by e‑mail, we have about 30 e‑mail to include and one they're done counting all the votes they'll let me know and I'll announce the winner. The voting sheets are being distributed. Everyone who is in the room can vote so you're welcome to vote everyone. Please help them distribute the voting sheets and cast your vote. While that is commencing I would like to go to the second part of the agenda which is the BCIX update. It's very quiet this morning. I wonder how that is, why that is.
SPEAKER: There is not enough voting sheets. Right. There's two ways to that. It's basically a simple one, two, three, so if you have a sheet of paper like a general white sheet of paper or someone can run and make copys. Maybe someone could run and get a number of white sheets. Still a few. Okay. So I suggest that someone, and I need a volunteer here, runs to the printer up stairs in the terminal room and gets a couple of white sheets. Wolfegang, thanks a lot. Both of you? Thanks.
BCIX update, tell me is it BCIX or B kicks
SPEAKER: I would like to give the ‑‑ /EUB ma the first introduction about the BCIX at the RIPE meeting. I work for BCIX since last year. Okay. First they were founded as an association in 2002. BCIX runs the Berlin internet exchange and we have current three such locations in about Berlin. We offer peering ports and the usual port speeds and currently we have about 3 gig of traffic. We're an euro‑IX member as well and the executive board consists of five people. This is mixed ‑‑ we have two technical people and quite a lot of finance and Helga for the communication. We have 43 members as of today. Members are quite different like small companies from Berlin, small ISPs, but we have big carriers as well as you can see.
We had a growth curve of about nine new members over the last 12 month. A little bit of loss.
And today we have 40 active peering ports. As you can see here ten gig ports, this was coming very late and that's because BCIX was a free exchange until last August, that means you only have to pay the association fee and peering ports were fee so we should not money to pay peering port so we introduced fees last August and we introduced new switches with ten gig ports.
We have three locations, one is Versatel, at that time they were owned. The second location is carrier‑COLO and the newest one is quite big, 181000 square metre space, have the possibility high power works there. We're proud of this. Technology, we have 4,000, 8,000 and a newer switch.
Between the locations we have darkfibres so we can scale the bandwidth and open BGPD in January.
Okay, here a map of Berlin, you can see the location. We have E shelter on the left and Versatel, which is near to the level three gateway, which is next door to this location.
Okay, route server, open BGPD, I think we'll speak about this later. We run this without problems since January and we have a couple of peers on it as you can see. We have communities but no filtering active on that route server.
Okay. We look for more peers, of course, as other exchanges also, and specially for CDN and content providers because we have a lot of excess networkers on the exchange and need the content to get more traffic.
Okay, if you want to join us, we have port fees, of course. This is 300 euro for gig port and 1200 euro for gig port and 600 euro membership association fee. Okay. That's it. Any questions?
CHAIR: No questions? Thanks a lot.
(Applause.)
Next up is Maurice if he's here. Yes. Some peers ‑‑ I forgot about that, I switched around the agenda from yesterday.
Maybe you could switch it around and do the other presentation first? Yeah, okay.
So in that case, you're up afterall. Lots of technical stuff today. Sorry guys. There you go.
SPEAKER: Good morning, my name is ZBYNEK, and I have an idea about a new function of the equipment of an internet exchange. Still nobody told me it's ‑‑ I would like to present it here.
As we are a user about ten internet exchange point in Europe, we have discovered one of the weakness of the shared medium in the internet exchanges is when a link to one ISP fails, all other ISPs knows about it after one or two minutes when the BGP session becomes timeed out. So how it's possible to solve this problem? It is possible by tuning of timeers of the BGP. What is the way to Hell or by BGP which must be implemented by all ISPs, so it's also hard because not every routeer supports it, not every ISP wants to support it, et cetera, et cetera.
So? My proposal, how to solve this problem is to implement a function to the internet switch in the internet exchange point to the add switch in the internet exchange point. When the switch monitors all BGP sessions goes through each customer access port and when the port goes down, when the port is broken, it takes an IP address of the broken routeer and sends a packet into all BGP sessions in the other end of all BGP sessions going through the affected port. I think it would be ‑‑ it could be understandable for you from the picture.
This is the ‑‑ very easy algorithm how it's possible, how it's possible to implement it into the internet switch, specially to asick or APG circuits in the line cart of the switch. This is the algorithm of learning of BGP sessions. It's easy, the only thing is the switch monitors, monitors all traffic if it's some BGP traffic and if it is BGP traffic, it handles it as describes on the picture.
The same algorithm is for the reset of sessions when the port wheel is down. So if the port goes down, the switch sends RST to all systems going through the affected port. It's really easy only the handling of TCP, ‑‑ of TCP segment number is necessary to handle for this solution.
So what is the advantage? The time between problem detection and reaction could be rapidly decreased from minutes to tens or hundreds of milliseconds.
So it's seemless for users when the problem appears, not as now when you are waiting and you see there is an outage.
There is also known problems. The first problem is somebody must implement this function into their boxes and the second problem, I know about is MD 5 protection. It could be solved by a database of FD pass words in the switch. It's not a good way maybe. It could be. I don't know. It's partly possible to solve it with using TTL security instead of MD5 pass word protecting.
So thank you. Some questions?
QUESTIONER: Steve Kent, BBN technologies.
SPEAKER: Yes.
QUESTIONER: In addition to TTCMD five there's ongoing work i4n the ITF for successor security mechanisms for TTC specifically for protecting BGP sessions and those are automatic which make it completely infeasible to have any passwords in the switch in addition to the fact that you're looking above several layers where the switch should look in the first place.
SPEAKER: Okay, thanks.
QUESTIONER: Stephen backer from AMS‑IX. The idea itself might be intrigueing but I don't how see how it's a going to work on a layer 2 network which involves multiple switches which will need to know all those peering details and how to do it on multiple 10 gig E lined rate. You have to look at every single packet to make sure ‑‑ to check whether it's BGP packet and take action on it. I can tell you already the current switches have a hard enough time to switch the traffic that they have to do instead of just inspecting every single packet and doing something with it. I really don't see this working in any feasible way. I'm not sure what problem you're trying to solve. The most problems is not timing out of BGP sessions but the excessive flaping of BGP sessions due to a short link flap, for example, and on those occasions you don't want your BGP sessions to flap, you want them to hold on for a short while. So it's just going to create more turn, I think, in the routing table than is necessary.
SPEAKER: Okay, thank you.
QUESTIONER: I think you won't get any switch runner to implement that actually. I think internet exchanges are a small fish for the big switch providers and every feature we want we usually have to fight for. And I think this is a feature that's really ‑‑ I think it would not need a lot of time and implementing stuff. So I think you won't get the switch runners doing that. You could have a server doing that. I think that connects to ‑‑ some kind of a weird hack or get traps and it would send out spoof messages but it needs some kind of interference with the switch anyway so I don't think it's working.
SPEAKER: Yes but doing it this way could be much much much slower.
QUESTIONER: Yes, of course.
QUESTIONER: Ruedlger Volk, Deutsche Telecom, abusing security leaks to sneak in stuff just get BFG ‑‑ okay, for those parties who do not want to support it long term, well okay, they may have ‑‑ they may have some peerings denyed because of parties want to have it, well, okay, people who are not up to state of the art have some difficulty.
SPEAKER: Okay, it's necessary to be possible to switch this function down on specifyed port, of course, like any other function including CDP or I don't know. (CDP)
QUESTIONER: Google. I would suggest to you to consider something like extension route server function alternate. When route server is part of the same exchange so you can get from switching platform of the information involved link failure and as you have a peering session, you may have some code extension on route server like the message prior to something. You can basically speed up operation of route server and link it to like link flop conditions, as an example.
And question to you: What is specifically bad in tweaking BGP timeers? Do you have any numbers to back that up?
QUESTIONER: I don't say it's bad (speaker) every manipulation of timeers, specifically down and ‑‑ specifically if on the session is a very, very large amount, amount of updates and prefixes. It's a lot of stress for the CPU of the routeer. And that is a lot of providers who are using software like Cisco and I don't know some bloggers. So it could cause flaping only because the Quagga doesn't ‑‑ can't important all prefixes in the time before the session becomes down. For example, it's my experience with it, yes? So I think it's not a good idea to making bigger stress for peering routeers.
QUESTIONER: Thank you.
SPEAKER: Okay. Thank you.
(Applause.)
CHAIR:
QUESTIONER: RIPE NCC. Could we just have all the ballot papers in in the next couple of minutes so we can count them during the session. Raise your hands with the paper if you still have one. One there, one in the middle. One in the back.
Okay, Maurice, you're up next or now actually.
So Maurice is going to talk about some developments in Paris.
SPEAKER: As we just sort this out, it's actually a quick presentation on not exactly an exchange but an idea of a proposal to investigate building an exchange switch fabric in Paris. We just ‑‑ I'm just going to talk without the assistance of slides.
I work for Google but I'm here under a personal title working for a ‑‑ working for a Working Group on a project we've called phoenix which is to look at the market situation in Paris and see if we can help address some of the short falls we see with the exchange environment there.
We'll start with what we see as some of the challenges. Paris has the fortunate position of being very central to European Telecoms industry. France has numerous submarine cables running across the Atlantic into the Middle East, into Africa and all the way into eastern Australia, with landing stations in south earn and northern France. If you look at the environment, the exchange environment within Paris, there seems to be a surprisingly few ‑‑ few large‑scale exchanges that represent the countries ‑‑ or have members from the countries that are connected via those submarine cables. Also there's something of a fragmented environment where I think at the last count there were 11 internet exchanges within the metro area of Paris. That may have increased since I last counted. They tend to appear and disappear.
In discussions with various networks or network operators and content networks, there's a clear view that this fragmentation, this numerous number of exchanges and the complexities of operating in Paris is a detriment to future growth. So we've formed a Working Group of sorts to take a look at this and see if we can formulate some proposals of how to address this.
Early conclusions are that there is a demand, that demand looks like a demand for a robust, well‑supported distributed switch fabric architecture within the Paris metro area with some features maybe not to be present in all existing exchanges, things like the ability for advance reporting features based on MAC addressing or accounting, S flow, the support for platforms or services that go beyond the normal standard exchange of IP traffic.
So, for example, platforms to support the interconnection of VOIP, maybe video conferenceing and other platforms. And we ‑‑ we're starting to think around how we can address this demand. Well, first of all, we think ‑‑ the structure we're thinking is that we should look at consolidating a number of the existing infrastructure, number of the existing exchanges that may be located in one or two facilities and see if we can't build a fabric in which the participants connected to those exchanges can exchange traffic with other participants in different buildings, different facilities.
The second conclusion we have is the formulation of this idea ‑‑ or the most successful of the type of models like this have been under the auspice of a non‑profit organisation and we think this is probably the best way to approach. And with an emphasis from an operations viewpoint and designing for scale and operating a level of technical and business operations that are focused around performance and availability of the infrastructure.
So that's a quick introduction.
We're trying to get a good understand of those demands in a meaningful way backed up with some data. We originated a survey that we're distributeing hopefully starting today, just asks a few questions around ‑‑ of those ISP facilities and CDBs and general operators who are present or intend to be present in Paris. What their requirements and demands are, the challenges they see. We're going to collect data over the next three weeks to see if we can't have ‑‑ make some sense of it and refine this proposal into this more detail. At that point we'll formulate a feasibility study and if the feasibility study doesn't say no, it's a ridiculous idea drop it immediately, then we'll look at the next steps of forming a commercial Working Group and a technical Working Group to take a look at the proposal in more detail.
That's where we are with the PHEON‑IX idea. If you could go into the website, which the address is here if you have strong opinions or comments, please leave them and we'll be grateful to read through them.
There's also ‑‑ before I open the floor to any questions, we'd also be interested in interesting a discussion with ISPs or facilities that have an interest in Paris. If they're outside of the format of the survey, we want to approach some of the Working Group to have an off‑line discussion. Please approach myself or Christian, we're all there or probably at the bar for the next day or so.
Yes, that's it. Any questions?
QUESTIONER: High. An observation rather than a comment or question. You might want to have a look at your survey and remove the German and perhaps put some French in instead.
SPEAKER: I didn't catch that.
QUESTIONER: Neither did I. Some of the words in your survey are in German, instead of others I see the German word. So your survey is partly in German.
SPEAKER: Well it's taken on a level of artificial intelligence and I'll be patenting the technology and will be available for licence. Thank you. I'll take a look at that. Any other indication questions or indications of my next business model?
CHAIR: No. We'll have a look at that survey, I guess, because that's interesting.
We have a next presentation about the secure information exchange from the ISC (ISC)
Is everything up loaded? No election result yet? Still being counted.
After that, we'll have the best common practices for IXPs and we'll close off with the route server discussion.
With
SPEAKER: Okay. Good morning. I'm Keith Mitchell. Little while since I spoke to anyone, living over in DNS land past few years. Nice to be back. I'm going to talk about a different exchange, working on SIE, Security Information Exchange. I'm sure you're familiar with the concept of exchanges as trusted intermediarys fending off the come pet terrors trying to get them work together. Internet exchanges have been around. There's other kind of exchanges for trading and also interconnect, not just the switching that we often do. What's the idea of a Security Information Exchange? As you all know internet security is big business these days but there are some limitations in the way that people deal with it. The idea of the Security Information Exchange is we can do exchange of security sensitive data in real time and also in a centralized multi‑lateral basis which is something that doesn't happen at the moment and use it as a way of fighting back at the bad guys. This is how security information is shared on the internet at the moment. Lots of different players, players have their own communities, various bilateral options but not a clearing house for this kind of information.
Here's something exchange operators are a little more familiar with, sharing information between the different parties. Everybody contributes. Everybody potentially consumes and analyzes the data coming in, the data shared. Obviously this is sensitive data so it's important to of a policy framework and a legal framework.
So here's the kind of thing we have at the moment, a bunch of sensors out there, passive DNS sensors, so gathering DNS query data but that's scaling up to more than just DNS stuff now. We have some relay machines, more about that in a minute. We have a switch, which is a layer switch 2 fabic but it's used a little different and we have a bunch of reserve doing analysis and summarizing the results of that back other to the exchange. This is in the San Francisco bay area. We're looking to expand this to other locations. We're in the process of building another exchange on the east coast of the US, probably DC and New York. Again another major data center where all the players are. And again there will be sensors feeding that. It will be possible to summarize data between the two different locations. We don't want to do full strength real time feeds because very large volume but some of the data potentially shareable between the other locations.
And then it also makes sense to potentially gather data from other parts of the data internationally and local relays, again to summarize that. And one of the reasons for having local relays is that ‑‑ depending on privacy legislation, limitations, you may not want to move the data between countries. Some of the data sharing you may want to keep within a particular country or EU rather than back halling between international boundaries and potentially causing issues. It's also possible to do analysis on the data, to make sure that personal identifiable information is not being shipped around between the different nodes, which is also important.
And then if you get enough capacity in a particular location, then potentially make that a full node. We're talking to various exchange operators at the moment and we'd be interested in talking to more and setting up locations close to existing peering exchanges.
So getting back to the model, there's a lot of data out there used in the abuse community, spam, DNS black list, passive DNS, log files, anti‑virus, anti‑fishing activities, abuse reports, honey pots, all kinds of things. We've put together some tools, Ncap is a tool you might find interesting, it's designed to parse DNS messages to quite a high level. Various plug ins which can be used for filtering level for broadcasting. We have generalized broad /K‑P. It takes a lot of hardware. One of the decisions we made is we don't store data. This is all in real time. That places interesting demands on the switching hardware, on the server hardware, it means quite important architecture decisions. Hard discs can't cut it. We need lots of ram, we need flash discs and we've been deploying a lot of that.
So NCAP is a derivative of something that was DNS cap. It does some things that are very useful, if you're looking at passive DNS data, it can defrag the queries, gets rid of link layer information as a normalized format, uses accurate time stamps and other features so it copes better, has modules and the deduplication is very important as well.
If you add together all the passive DNS features we have, which are queries from up stream of catching resolving, that's a very large volume of data. So these go straight on to the exchange but there are analysis machines ‑‑ some researchers want to look at all the DNS messages and others are interested in other things, we do deduplication. There are various filters that we can do, for example, so that you see changes between DNS queries for people researching Fast Flux used for controlling botnets. But yeah, we're seeing 201000 packets per second just from passive DNS traffic. That's a lot. If you deduplicate it you get it down to 31000 packet which is easy. If you start running filters on that, so you're only looking at changes, then you have the ability to do Fast Flux almost human eyeball speed, try and figure out who's using Fast Flux DNS to run their botnet.
A little bit about NMSG, where's the PCap, DNS cap? Format was basically a way of rebroadcasting DNS queries. This NMSG is for a whole bunch of different types of structure but still using a broadcast exchange but these can be things like spam headers, used as a generalized protocol buffer format which I won't claim to understand but this is something Google has come up with and seems to be a useful tool for generalized handling of data. Something you don't see often, fragment of code. Don't ask me what it means but it does the necessary stuff.
The way it works is we have a bunch of channels which correspond to V‑LANs, most of these [are] Mac leer broadcast. You wouldn't want to run it over your peering fabric. It needs dedicated switches. We've talks about passive DNS, Fast Flux, various channels for anti‑spam black list, various types of DNS traffic, polar sites TLDs, so you can find the people who are sourceing the PTR look ups that shouldn't be. Various legacy root which are no longer author tiff or used but seek a huge number of queries. Lots more channels. We want more spam. You recall link pairs, net flow data, various sink holes and other things. SIE has proved a grate tool and there's some quite interesting analysis being done.
Potential queries from dark net space, taking chunks of space and looking at what's coming in and also various malware activities as well. One channel example is spam. You can get it from spam traps, supports, and obviously it's not just spam. You can use what you get out of spam analysis to look at other bad activity but the spam is trying to trap people into malware or fishing. How you should represent an e‑mail message.
Mentioned malware analysis. Some people can submit mal wear, provide links to it. There are various tools, these are all in real time, analyzing that. Some people can share data. Other people find this a valuable use of raw data. And there's lots of other things, reputation analysis, DNS scans, the results of going out there and trying to fingerprint all the DNS servers, finding out what version they're running, maybe we can use it for DD OS reporting and a whole bunch of people are going to the same you recall, maybe it's an event or may be it's a malware, maybe we can use it for BGP traffic.
So this is a collaborative effort. If you can help with this we'd much appreciate it. We're always looking for data. Once that data is there, there's the possibility of sharing it. If people want to bring servers, we can host them. And the other thing that's helpful, particularly from ISPs, particularly if you're running large or cursive name servers, is to install sensors and get this data so it can be back halled into SIE and analyzed and it can be used to protect abuse on your network.
How does this work for exchange operators? Another type of value that you can put into your exchange. It's not just about exchanging raw data but about doing analysis on that data. You can, for example, work with your ISP members and help them deal with abuse, which is always a problem, and there's always potential in new tools. You can do outreach to law enforcement. Be more practical about security generally. But obviously the other thing you need to be careful about is privacy laws and policies. This isn't specifically wiretaping but it's looking at data and it's very important to make sure that data is dealt with appropriately.
So that's about it. Our program manager for SIE is in the Jabber room at the moment. If there's any questions I can't field, Eric will be happy to field them, either online now or if you want to contact him afterwards. Happy to take any questions.
No questions?
CHAIR: One question.
QUESTIONER: Hi. Eric from UCLA and CSU. As a researcher, how can I get access to some of this? Specifically the DNS names?
SPEAKER: You need to sign an agreement to make sure the data you're receiveing and analyzing is dealt with appropriately and maintains confidentiality and we can provide you with a server you can plug into and you can tune into one of the channels, passive DNS channel and it's up to you then what analysis you do on that
QUESTIONER: So just provide a box after signing an agreement?
SPEAKER: Pretty much, yeah.
CHAIR: No other questions for Keith?
SPEAKER: Okay, thank you everyone.
(Applause.)
CHAIR: Letter. Then we have a presentation by Wolfe gang, it's about best common practices in IXPs.
SPEAKER: Hello everyone. My name is Wolfgang. I'm talking about IXPBCPs, this is a project that arises from the euro‑IX community. It was originally started by the Italian guy. How was he called? Clare
CHAIR: Simone.
SPEAKER: Yes, he started this and after he left the Italian IXP I took on with that project. It's basically a document for technical practices for internet exchanges. So it goes through set of maintenance of an exchange point infrastructure or ‑‑ you also get a set of recommendations from already established IXPs. So what you basically get there is information how to set up or run an IXP. If you haven't got an ‑‑ if you want to create an IXP, just do it and follow this best current practices document. You will get an IXP that is more or less the recommendation from our community. If you already run an IXP and you want to know how do others do monitoring, how do others have, like, the set up for out of band management and things like that, you will find it in this document. It's not described in whole detail, how do you set up whatever, but it generally tells how most of the IXPs in Europe or members of the Euro‑IX community do that. What it does not address is commercial or political communities.
This is not a commercial for Euro‑IX but they get access to sample configurations from IXPs some contribute some this won't be published because the IXPs decided not to publish their internal /KOB tent for security reasons. They also have the possibility to contribute content. And they have access to more than the general IXP guide Lynn lines. There's a website, there's an up coming route server, or whatever.
So this document will be published on the Euro‑IX website soon. There's still some time for Euro‑IX members to read through it before we publish it. There might be a possibility to add comments for non‑members. Maybe it's just done by e‑mail, if the request isn't too high or a comment section on the web page. But if you have a friend who wants to create an IXP or know how things work, this is a good document to address that.
And I think this is it. Yes. Any questions? No questions.
QUESTIONER:
CHAIR: No, wasn't a question. Okay. Thanks, Wolfe gang.
Before we move to the route server discussion, we have the results of the elections.
SPEAKER: I've counted the election and the number of ballot papers is also double checked by another member of the staff. Quite close run. We had 110 ballot papers from the room. We had 28 e‑mail votes. I've discounted five additional, that's 28 valid. There were five invalid because they came in after 10 o'clock. The last one came in a couple minutes ago. So 138 valid votes all together. In the first round we had 57 votes for Andy Davidson, 36 for Wolfgang and 45 for Remco , so we've taken the votes for Wolfgang and redistributed those and the final result is Andy Davidson 68 votes and Remco 59. So I declare that Andy Davidson is the winner of the election.
CHAIR: Congratulations Andy. Will you come up. I'm just inviting you up here and you can make a...
Andy: Thank you very much.
CHAIR: You'll be up here next time, in Lisbon, I guess.
Okay, very well. So to close off, we still have about 25 minutes. We're doing some discussion on route serving. I would like to close a little bit early, like five minutes, so everybody has enough time to get to the lunch and I have to go to the Working Group chair lunch. So we'll all be happy and early for lunch.
I would like to invite nick who wanted to kick off the route server discussion with a couple of slides.
SPEAKER: Good afternoon everybody.
I want to explain why a lot of internet exchanges use Quagga for a route server. It's not always obvious. There are a lot of BGP stacks out there. The problems start arising when you have a situation where you have multiple members on your exchange and you have ‑‑ do you have a pointer? ‑‑ where you have a requirement to do prefix filtering. Now, not all of the commercial BGP stacks like Cisco and like that like stripping out your ought none mouse number, Quagga and BGP bird will do those. Quagga supports multiple ribs. To explain why this is a good feature I'm going to use this diagram here.
You have here four route server clients, A, B, C and D. A and B are providing connectivity into the exchange for these clients over here, X and Y. So you can immediately see that this path here AYX is longer than BX so you would expect that B would provide connect tiff into the exchange for X. So that's what's going to happen on a single‑rib route server. It will run it's best pass calculation and calculate that the best path is indead BX. But in real life when you're running an internet exchange route server you really want to provide the facility for clients of that route server to implement filtering, outbound filtering from the route server so other people will not necessarily be able to see your prefixes. The usual way of doing this ‑‑ and there are others, and this isn't ‑‑ the problem here isn't specific to using BGP community tags but on a lot of exchanges we use the notation ‑‑ we type prefixes with 0: , whatever the AS number that you don't want to see or receive your routes and then the route server will filter those out. So B here has tagged owl of its prefixes with 0: D because it wasn't want to appear with D for whatever reason, personality conflicts or commercial interests or whatever. So what you'd really like is D would be able to get back to here using this back up path of AYX but it can't, the route server has only a single rib and single best path and that path is BX and therefore D cannot see this autonomous system number over here through the route server.
Quagga gets around this by implementing multiple loc‑RIBs. Very quickly, you have A, B, C, D, are route server clients. Same situation as before, AYX is longer than BX. Now, the route server is different now, you have multiple ribs inside it and in order to implement internal filtering within the route server Quagga implemented export and import route map filters, they're a little bit like in and out except that they work internally.
Okay, all of those prefixes are going to go into all of these lock‑ribs here, prefixes from A are going to be into ABC and prefixes from B are going into BC and D.
So RIB D is only going to see YX, not BX because BX specified it doesn't want the route server to transfit its prefixes to D.
So RIB C will select the best path to being BX here and RIB D will see the best path as AYX.
So that's why you kind of really need multiple Loc‑RIBs. Quagga at the moment is the only easily available BGP implementation with Loc‑RIBs. There are a couple of others not easily attainable.
There are a couple problems. With a regular BGP implementation your resources are linear. If you have a million prefixes, that's going to be ten times as much work as 100,000 prefixes. With multiple lock it's going to go from P to that, which is order N squareed. And that's really horrible. So that's ‑‑ basically if you do on to show that summary, you multiply the number of prefixes received from each client by the total number of clients and that's roughly the order of magnitude of complications for memory and CPU. This is immediately a problem for any BGP implementation. Memory is fine, you can bang in heaps of memory and that will keep you going. But really you need a multi‑threaded BGP implementation. If you do everything on a single core and if you have piles of prefixes and piles of clients like some of the internet exchanges they're going to get very busy. It's fine if you're small they have no problems but the others it hurts.
So that's a brief introduction into why a route server at an Internet Exchange is a little different from the regular BGP stack.
Any questions?
CHAIR: Thanks for that introduction. So that was about a Quagga set up. Now, I've talked to a couple of people here in the room that also run route servers and that have specific set ups, specific choices that they made and they would like to say something about that and why do they tolerate or chose for what they chose for.
Andy, you want to go first? You're building one, you are building two actually?
SPEAKER: I hinted at this yesterday. We want to build route servers because the evidence we've seen around ‑‑ when we discuss this with our colleagues at other exchanges, it's a good way of growing value for people as soon as they connect to the exchange. It's a service that a number of people have individually asked us for and it's ‑‑ it's a good way to grow traffic which helps with marketing. So we've decided we want to build some route servers but didn't want to build another Quagga because there were a bunch of them in London. We're going to ‑‑ we're likely to roll out route servers on open BGP and support the AMS‑IX's to add per peer RIB to that software, and also roll out Bird, which the guys from Prague are doing a lot of work to make good ‑‑ I think we're going to be the first exchange to role that as a route server. We're working really closely with those guys to swap really good feedback. If there's other exchanges in the room making similar decisions, we'd certainly swap information about how successful the new builds are going. That explains our position and why we're not using Quagga.
CHAIR: You mentioned two others. If anyone has a remark or question in between, go up to the mike and state your question and remark.
Now, Andy mentioned AMS‑IX as working on a new set up for BGP.
SPEAKER: We're moving away from ‑‑ we're currently using less Quagga, it's just basically too unstable, it really has scaleing problems and we really see the single‑threading issue. So we're going to ‑‑ we're actually moving to which isn't quite ready yet (BGPD) we're paying developers to put that in and hopefully we're going to see something. We've seen CVS commits on the open repository. Something is happening we're just not really sure what's happening now. It's in development. So...
CHAIR: I think Andre is going to talk about Bird a little bit here.
SPEAKER: Thank you very much. Andre Phillip and as was mentioned, sort of group people, the devil of the Bird, quite an old one but not very well known, and we are currently running two Quaggas as a route server and are not happy. In any case we would like to have solutions so we are working on replacing one of the Quagga by Bird. And many of you probably doesn't know Bird. It's a routing demon which is developed by people who have close relationships with this exchange point. Like, for example, Nick presented those will be hard and the people understand the problematics of those things. So if you need some more features into Bird, just let us know and we can implement some new things into that.
CHAIR: Mike.
SPEAKER: I think one of the things I'd like to get across, we run right now Quagga as I described yesterday and I showed you some stats and number of peers and prefixes we receive on there. We ran Quagga when we first because really that's all there was. There wasn't much in the way of viable alternatives. Route servers have become very vital, I think, to the larger ‑‑ to a lot of the larger exchanges because if you think of an exchange the size of AMS‑IX or links where you have 31000 members ‑‑ if you have an open peering policy trying to maintain that number of peering sessions and that number of peering relationships on a configuring per device per session it's quite a lot of work and you have to ask yourself, if it wasn't the route servers would that peering community and environment be so rich? And probably wouldn't. People would say I'm not going to peer with these hundred people because it's not worth the time, if I can get it via the route server it's low cost and easy pickings.
We're currently in the situation where we have issues with Quagga, hit some of the scaling issues that nick was just talking about, and we're basically just backing several horses in this race because there are lots of alternatives, although there's starting to be a chance of lots of viable alternatives, Bird one of them, open BGPD once it has the ‑‑ (BGPD) but there wasn't that many alternatives, so I think what we're seeing is, what we know is have his /SR‑S a vacuum and it's trying to be filled and there is a vacuum with Quagga with the larger exchanges and we're trying to fill that gap one way or the other. That's just my observation of the state of the nation with route servers at the moment. We're just trying ‑‑ basically done some things to buy us some time with Quagga but that's all we can do until we find a replacement and that's spending a lot of time and effort on that at the moment.
CHAIR: Any other exchange? Harold, go ahead. You're also in an implementation phase or thinking phase.
SPEAKER: As I mentioned yesterday in the introduction, we currently plan to implement a route server as fast as possible but besides the decision which software to take we're intent to take Bird because it's developed by IX people and there's always rhetorical questions, how do we implement the filtering issue and Nick showed it's common to use BGP community text to give the route server information which prefix has to be announced to which peer but this is not compatible with 32‑bit models and we want to build a new route server which is capable with all and I don't want to call 32 numbers a problem but if you build a new route server it should be compatible with everything that could come now. That's our main problem. We want to build something that is able to do all our requirements at the moment.
CHAIR: Anyone? Arnold? Yes.
SPEAKER: Technically, there's not much I can add. Almost everything was said. Currently we at DE‑CIX are still running a fairly old version and quite stable compareed what I seen at other exchanges.
I guess I should also mention that last week at Euro‑IX we talked about it and we decided to form a Working Group on improveing Quagga and we already see substantial improvement in gathering resources both in personal as well as collecting money, and I'm quite sure that next time in the RIPE meeting in Lisbon, we'll be able to provide results on this.
CHAIR: Thanks for mentioning that and I'll save you a lot then.
Okay, I think that concludes. Unless there's anyone else that wants to say something? Okay. That means that we're done for today. I would like to thank you all for being here. I would like to thank Fergus for being remote. He will be here the next time or at least we'll see about that. Andy, congratulations, Wolfgang, thanks for standing, Remco , thanks for standing. Big applause for those guys as well.
(Applause.) See you next time. Roland, also thanks for counting the votes. We're done.