| By Basil Voronkov at 2008-12-19 |
Starting from Beta 2 WideStream supports custom extensions. Moreover it is possible to add a support for a new protocol (by default only HTTP and FTP are supported) without the need to make any changes to the rest of the application. Also you may implement custom download adapters (see previous blog entry for details) as well as custom browser integration providers.
Basically all you need to do is to create a manifest.xml file and place it in the same folder where WideStream is installed. By default an application has an embed manifest that includes configuration of HTTP, FTP protocols and other standard functionality. You can take a look at the standard manifest in the project repository . When you create a custom manifest.xml file in the application folder it is processed in addition to the standard one.
For now manifest has three sections - Integrations, Protocols and Adapters. Inside each of the sections you can declare your own add-in by providing add-in name, description, a fully qualified name of the implementation class and, depending on the add-in type, a fully qualified name of the add-in options class.
As you can see native WideStream functionality such as FTP download support which is present since the very first public release is now being "added" to the application as an add-in. (By the way you can find a list of all active add-ins using a new Add-Ins dialog accessible from Tools->Add-ins menu).
What this functionality gives? For example, WideStream can't download from your favorite site (that requires to perform some actions such as submission of HTML form with some site specific information prior to download) - there is no problem at all, just create your download adapter as it is done now for the VBulletin attachments or YouTube video. WideStream doesn't support integration with your browser? You can teach it how to integrate with it.
Moreover if you are planning to implement your own network application - for example an SSH/SFTP client - you don't have to create all the application from scratch. You can use all the functionality that is already implemented in WideStream such as configuration engine (user interface for your settings is being generated dynamically based on your Options class), host manager, download queue management and much more. You can concentrate your effort on the SSH/SFTP client implementation and save up a lot of time. |
| By Basil Voronkov at 2008-12-16 |
In most cases download process is pretty straightforward – you are requesting a specified file from a specified host and start to read it in chunks in a single or multiple connections. Certain servers however could be more demanding. They might require you to authorize using their own specific authorization technology or to set some special cookies or even to maintain an opened session while downloading a file.
There is a number of ways to solve these problems – the most popular one which is used in most download managers is to obtain all specific download details (such as which cookies to set) through the browser integration.
What does it mean? For example, you see a link on a web page in your Internet Explorer. You have navigated to this page using your browser which means that you already authorized even if you didn’t enter any credentials – the host have sent to Internet Explorer everything it needs to open a user session, your browser has memorized that and now when you click a download link it will send all these “authorization details” as we can call it prior to starting download. That is why it works.
Try to copy this link and manually add it to your download manager. Does it work? Nope. Why? Obviously because your download manager is not a browser - it just requests a file, it doesn’t communicate with the site, asking it which information needs to be set, how to authorize and so forth.
And now – assuming that your download manager is integrated with Internet Explorer – try to download the same file by right clicking a link and selecting your download manager or even, depending on the integration technology, just by clicking a link directly. Do you see the difference? Of course, now it works. Why? The answer is pretty simple – Internet Explorer tells your download manager – hey, man, you need to set up these cookies. And that is what download manager does.
Sounds like a good solution? But in fact it is not as good as it sounds.
- First of all it only works if you click a download link in your browser. It is not really obvious even for me – from a user standpoint – why the hell it works when you click a link in a browser and doesn’t when you click a link in IM sent by your friend? Does it mean that we need an instruction on how and where to click links so they will always work?
- WideStream starting from Beta 2 also has a support for browser integration (at this moment only Microsoft Internet Explorer 6 and 7 and Opera 9) but browser integration should always be integration – it shouldn’t change a behavior of your download manager. It something doesn’t work than it doesn’t work. If it works than it should always work.
- And now the headshot – your download manager obtained cookies from your browser, starts to download, open multiple connections, shows you that resume download is fully supported – everything seems to be perfect except of one thing. The information provided to your download manager is likely to expire after a short period of time (usually 20 minutes). If you pause your download and try to continue it later... well, you will have to start it from the very begging. Because you download manager doesn’t know how to obtain all the authorization details, it simply parasitizes on your browser. And of course you should know about that in advance – your download manager may pretend that pause feature is supported, don’t believe it! It is not always true. Probably we also need a special instructions in which cases we really can pause and in which cases we just pretend to be able to do it.
As for me these are pretty serious disadvantages. Finally all this functionality – which is I believe only semi-working – might be completely broken if you’re using a browser that is not “supported” by your download manager. For example, I am playing with Google Chrome for the past several weeks and most of download managers can’t integrate with it.
Of course after all this criticism it is reasonable to ask – and if there is another way, a different approach without all these annoying contras that we can offer? Fortunately the answer is Yes.
As you can notice we come to all these difficulties because of an assumption mentioned at the very begging of this post – that download manager is not a browser like Internet Explorer. And why not? Of course it doesn’t mean that we should display the web pages and force user to navigate between them before starting our download – but who says that we can’t authorize against a site if a site requires us to do so?
- It will work in all the cases – when you click a link in a browser, when you click a link in IM, when you don’t click a link. It doesn’t matter, it just simply works.
- Guess what? You can pause and resume your download in a day or in a week (of course if a web server supports that).
In the cases when a site requires you to enter credentials in an input form – in other words when some sort of user interaction is needed before a download can start – we can implement site specific handlers that can do all the job. These handlers can automate authorization process, limit the maximum number of connections that can be opened for this site (which will override all other maximum connections settings), set other site specific cookies and so forth.
That is one of the most interesting features along with download acceleration that will be introduced in Beta 2. We call it download adapters. More over this feature will be extendable – so you will be able to write your own download adapters for WideStream or to install adapters developed by other people. And of course a default adapter will try to handle all the requests in a browser way as well.
|
Project Blog 