In most cases download process is pretty straightforward – you are requesting a specified file from a specified host and start to read it in chunks in a single or multiple connections. Certain servers however could be more demanding. They might require you to authorize using their own specific authorization technology or to set some special cookies or even to maintain an opened session while downloading a file.
There is a number of ways to solve these problems – the most popular one which is used in most download managers is to obtain all specific download details (such as which cookies to set) through the browser integration.
What does it mean? For example, you see a link on a web page in your Internet Explorer. You have navigated to this page using your browser which means that you already authorized even if you didn’t enter any credentials – the host have sent to Internet Explorer everything it needs to open a user session, your browser has memorized that and now when you click a download link it will send all these “authorization details” as we can call it prior to starting download. That is why it works.
Try to copy this link and manually add it to your download manager. Does it work? Nope. Why? Obviously because your download manager is not a browser - it just requests a file, it doesn’t communicate with the site, asking it which information needs to be set, how to authorize and so forth.
And now – assuming that your download manager is integrated with Internet Explorer – try to download the same file by right clicking a link and selecting your download manager or even, depending on the integration technology, just by clicking a link directly. Do you see the difference? Of course, now it works. Why? The answer is pretty simple – Internet Explorer tells your download manager – hey, man, you need to set up these cookies. And that is what download manager does.
Sounds like a good solution? But in fact it is not as good as it sounds.
First of all it only works if you click a download link in your browser. It is not really obvious even for me – from a user standpoint – why the hell it works when you click a link in a browser and doesn’t when you click a link in IM sent by your friend? Does it mean that we need an instruction on how and where to click links so they will always work?
WideStream starting from Beta 2 also has a support for browser integration (at this moment only Microsoft Internet Explorer 6 and 7 and Opera 9) but browser integration should always be integration – it shouldn’t change a behavior of your download manager. It something doesn’t work than it doesn’t work. If it works than it should always work.
And now the headshot – your download manager obtained cookies from your browser, starts to download, open multiple connections, shows you that resume download is fully supported – everything seems to be perfect except of one thing. The information provided to your download manager is likely to expire after a short period of time (usually 20 minutes). If you pause your download and try to continue it later... well, you will have to start it from the very begging. Because you download manager doesn’t know how to obtain all the authorization details, it simply parasitizes on your browser. And of course you should know about that in advance – your download manager may pretend that pause feature is supported, don’t believe it! It is not always true. Probably we also need a special instructions in which cases we really can pause and in which cases we just pretend to be able to do it.
As for me these are pretty serious disadvantages. Finally all this functionality – which is I believe only semi-working – might be completely broken if you’re using a browser that is not “supported” by your download manager. For example, I am playing with Google Chrome for the past several weeks and most of download managers can’t integrate with it.
Of course after all this criticism it is reasonable to ask – and if there is another way, a different approach without all these annoying contras that we can offer? Fortunately the answer is Yes.
As you can notice we come to all these difficulties because of an assumption mentioned at the very begging of this post – that download manager is not a browser like Internet Explorer. And why not? Of course it doesn’t mean that we should display the web pages and force user to navigate between them before starting our download – but who says that we can’t authorize against a site if a site requires us to do so?
It will work in all the cases – when you click a link in a browser, when you click a link in IM, when you don’t click a link. It doesn’t matter, it just simply works.
Guess what? You can pause and resume your download in a day or in a week (of course if a web server supports that).
In the cases when a site requires you to enter credentials in an input form – in other words when some sort of user interaction is needed before a download can start – we can implement site specific handlers that can do all the job. These handlers can automate authorization process, limit the maximum number of connections that can be opened for this site (which will override all other maximum connections settings), set other site specific cookies and so forth.
That is one of the most interesting features along with download acceleration that will be introduced in Beta 2. We call it downloadadapters. More over this feature will be extendable – so you will be able to write your own download adapters for WideStream or to install adapters developed by other people. And of course a default adapter will try to handle all the requests inabrowserway as well.
