Welcome to McFunley.com Sign in | Join | Faq

N-Tier Remoting Frustrations

posted on Monday, February 21, 2005 2:05 PM by mcfunley

Link.

 

People DO want n-layer, because that provides reuse, maintainability and overall lower costs. Logical separation of UI, business logic, data access and data storage is almost always of tremendous benefit.

 

People MIGHT want n-tier if they need the scalability or security it can offer, and if those benefits outweigh the high cost of building a physically distributed system. But the cost/benefit isn’t there as often as people think, so a lot of people build physical n-tier systems for no good reason. They waste time and money for no real gain. This is sad, and is something we should all fight against.

 

I think I agree with this in total.

 

The project I work on has hit basically every problem with Remoting that exists.

 

  • Unbelievably slow dataset serialization (see KB829740). Kind of a big issue for us because many of our developers aren’t so comfortable with creating their own data structures / objects, and consequently were relying very heavily on ADO objects. I resolved the speed issue by implementing my own formatter sink.
     
  • SocketExceptions (specifically, WSAENOBUFS) when transporting huge amounts of data. To some extent this is mitigated by better hardware. Although Q322975 refers to this as a “BUG,” we spoke with the Remoting development team through our premier support guys and they refer to this as designed behavior. There isn’t a quick fix (ie, registry change) here, so I’m going to have to try putting in a compression sink to get us some headroom.

  • The lack of a default remote call timeout. This killed us in production because of 1) a network problem and 2) some bad codes that circumvented the HttpRuntime’s executionTimeout in a very unfortunate way. It took us 4 horrendous days of the servers crashing every 20 minutes with three Microsoft CPR people on-site to finally figure out what the problem was.

 

There are probably a few others which I am forgetting now. I am still talking to MSFT reps about the second, but I have no expectations that that will get anywhere.

 

But getting back to my main point (yes, I had one). The really frustrating part about this for me is that I don’t even consider our middle tier to be necessary. We put it there primarily for security purposes, and I don’t know, maybe we get that. But with respect to the way most of our developers write code, this is no help. It doesn’t necessarily create a logical service boundary, because nobody really gave them the direction to do that. To an extent it just makes the spaghetti worse.

Now for my secondary point. A lot of this is certainly our fault. But what drives me crazy is comments like this one:

 

This does assume you listened to advice from people like Ingo Rammer, Richard Turner and myself and avoided creating custom sinks, custom formatters or custom channels. If you ignored all this good advice then you’ll get what you deserve I guess…

 

It’s frustrating as hell to be backed into creating workarounds for lousy performance which you (well, your boss, who mandated this architecture) were never warned about and then to be openly mocked for doing so by the ‘experts.’ If we had known that Remoting would be throwing us this many problems, we probably would not have tried to use it for n-tier, and maybe with a little luck we would have decided against our flavor of n-tier altogether.

 

Get us some better advice, folks.

Comments

# re: N-Tier Remoting Frustrations @ Thursday, March 17, 2005 8:26 PM

Dear Dan,

To clarify Richard's (and Ingo's) comments, our Remoting advice is to avoid developing application code as custom sinks / custom formatters only because that application code will be hard to migrate to Indigo. If you are utilizing the Remoting infrastructure to tune your particular application, it's not inappropriate. I am sorry if the advice comes across (or is sometimes replayed) as mocking -- people sometimes get really excited about Indigo. ;)

With regards to your SocketException issue, I've actually been looking into this exact issue. As you probably know, this basically stems from the fact that our formatter can't serialize an object in a streaming fashion -- it serializes the object into a buffer. So, the information you're passing around is actually being serialized into a buffer before even being written to a socket. The problem you're encountering is that we try to write that buffer to the socket in one call. So sometimes we try to, say, write 64 MB to a socket at once, and we get back a WSAENOBUFS error that gets bubbled up to the user.

Empirically (at least with my experiments), you tend to hit this when trying to send a pretty large amount of data (many 10s of MBs, say). While we are investigating the feasibility of fixing this, fixing this would likely end up causing the other exception mentioned in the KB article -- an OOM exception -- to evidence itself. If you're sending around a single object whose efficient serialized representation is 64 MB, then chances are its in-memory representation is not that far off. For many clients, a 128 MB commit charge, even if ephemeral, is a pretty big ask. Situations vary, of course, but this has the potential to lead to sluggish performance or OOM exceptions. On the server, asking a method invocation to take 128 MB also has the potential to limit scalability or cause OOMs.

While we're continuing to investigate the feasibility of a fix, the advice in the KB article (http://support.microsoft.com/default.aspx?scid=kb;en-us;322975) really is the advice we'd give people -- consider passing around smaller amounts of data or passing them around in a streaming fashion. Our advice for "Indigo" apps will be much the same -- if you're sending around large amounts of data, consider streaming them or passing them around in smaller chunks. While I'll make sure "Indigo" doesn't have this same problem (I just sent mail to the appropriate QA person), it'll still be the case that deserializing a huge object will have the potential to be slow or cause OOMs.

I'm sorry this is frustrating, but hopefully the explanation makes sense. If it's any consolation, we do actually consider feature requests / bug reports like this quite seriously.

Cheers,
-mike

Program Manager
Distributed Systems Group (e.g., .NET Remoting, "Indigo," ...)
Microsoft Corporation

  Mike Vernal

SKIN NAME : ImageHeader