Tag Archives: HTTP

How to Copy Large Files over VPN or Other Unreliable Network Connections

Network Error While Copying

Large file transfer over VPN is a problem for many companies for a few reasons, transfer is unreliable, VPN traffic kills the Internet connection, and it is unproductive. This article will show you how to copy large files over VPN or other unreliable network connections, and some of the best software to deal with this, and best practices to deal with large file transfer, and how to ensure file integrity. These are, in my experience, the best way to do it. You have to evaluate yourself, if they work in your environment, and test a lot.

When copying files over VPN, there are a few problems that need to be addressed:

  • file transfer can be easily interrupted,
  • over saturating the VPN connection,
  • redirecting all the available traffic to the VPN connection
  • ensuring the transferred file is undamaged

Let’s talk a bit about all of these, why they are important, and how they affect the success of your file copy. This will give you, hopefully, a clearer image of the process.

Network Interruption – File Transfer Failed

When we talk about large file transfers, an interruption after a few hours of transfer is not a good thing, and if you take in consideration the time spent, and to only realize you have to do it again because the transfer failed. There are many reasons for the network to interrupt, and even a second is enough to corrupt your file. VPN is prone to network interruptions with large files, because it saturates the Internet bandwidth, and when other VPN clients try to use the VPN tunnel, the file transfer will be interrupted.
The fix for this is some kind of transfer resuming at both the server level and the client level. A few client-server protocols that support file resuming are: SMB, FTP, HTTP, rsync.

The easiest way to implement file transfer resuming in a corporate environment is through SMB and robocopy. Robocopy has a feature to copy in restartable network mode, so if the network goes down, it will automatically resume the transfer from where it left. The robocopy option to copy in restartable mode is /z as with the regular copy command.

robocopy /mir /z  X:\source-folder\ \\RemoteServer\RemoteFolder

Rsync has also a resume option, and it works great, but you have to make sure you use the network filesystem and not a locally mounted network directory. The command to transfer will be like this:

rsync -aP juser@server:/RemoteServer/Directory /Home/Local-Directory

Filezilla has an option to resume files after interruption, but there is a timeout setup by default. Make sure you set the timeout to 0, so that you can recover even after a few hours with no connection between server and client. Note, that this is not a good option if you have many clients, since it will keep open connections indefinitely.

Time Out Config for Resuming

Apache has also an option to allow file resuming, which is enabled by default. The disadvantage with Apache is that, by default it doesn’t support file upload. If you are determined to use Apache for this though, there are some ways to do it, you can start your research here: File Upload plugin for Apache

Over Saturation of the VPN Connection

The VPN connection is there for many users, don’t think that your file is the most important think in the world. If you take all of the bandwidth for your file transfer, other users might not perform daily important tasks.

The over saturation of the VPN connection can be avoided by implementing bandwidth limiting at the software level.

You can do that with SMB by using robocopy, at the client level. The command will look like this:

robocopy /mir /IPG:250  X:\source-folder\ \\RemoteServer\RemoteFolder

The IPG parameter is the one that controls the bandwidth; it’s the acronym for Inter Packet Gap, and with an IPG of 250, the transfer rate on a 100 MBps network is around 12.7 MBps. The lower the IPG, the higher the bandwidth saturation. The transfer rate will be different for different network speeds.

With FTP is going to be at the server level, it’s very simple if you use Filezilla FTP server. Just restrict the bandwidth to a safe limit, (note that there is no restriction in the picture).

FileZilla configuration

Apache uses mod_ratelimit to control the bandwidth of its clients. For more information about that take a look at this page:
http://httpd.apache.org/docs/trunk/mod/mod_ratelimit.html

Rsync can also limit the bandwidth at the client level, and the option will be “–bwlimit”. A command to synchronize two folders with rsync, throttling the bandwidth, would look like this:

rsync –bwlimit=3000 /local/folder user@RemoteHost:/remote/backup/folder/

The 3000 means 3000 kbps. IMPORTANT, if you use

Control VPN Traffic

It looks like a simple decision to allow all the traffic possible through the VPN, most companies will determine that VPN traffic has the highest priority. However, in real life there are many non VPN applications that are run from the Internet, and are critical for a business. Booking a flight ticket, using a hosted web application, getting your emails from a hosted email server, etc… So it might make sense to control the maximum bandwidth used by the VPN, and this is especially needed in environments where large file transfers over VPN are very common. The best way to control this is through firewall policies to limit the bandwidth for the VPN destination. On some devices, like the Fortigate firewalls I am using, this is called traffic shaping.  On other devices might be named differently.

File Integrity Verification

There is a mechanism for checking the file integrity with robocopy and rsync.

The perfect tool to make sure your file is identical with the remote one is to check with md5. On Linux this is a package that comes by default in many distributions, on Windows you can use WinMD5, that you can download it here: WinMD5

MD5 checksum

How To Effectively Manage Google Plus Circles

For businesses present on Google+, or for authors, or for anyone who has many followers on Google Plus, managing their circles can get very complicated, and after a certain number of people in their circles, can even get unmanageable without a software tool. This page will guide you through many of the tasks needed to s
uccessfully manage your circles, will give you some tips and tricks, and present you advanced tools, to take this even further.

Simple Guidelines To Manage Google Plus Circles


Google-Plus-CirclesWhy Do We Need to Manage Google+ Circles?

Here are some basic rules to manage Google Plus circles, and these can be easily performed without any software tool.

Manage Effectively How You Share Content

Google+ circles are just a way to organize the people you want to connect to. It allows you to place these people in circles, based on relationship, (family, friends, coworkers, clients, etc…), based on interests, (web writers, pet lovers, art lovers, music fans, etc…), or any other criteria that makes sense to you. Think about circles as a way to place your social contacts into relevant containers. The concept behind circles is to easily choose what to share, with whom, and in what way. You can share content with certain circles only, or with the public. When you share pictures with the public, they can be seen by everyone, and reshared. When you share pictures with circles they will only be shared with that specific circle, and even better, they can’t be reshared with the public. That allows a bit more control of who sees private pictures, that you don’t  necessarily want to be spread over the Internet, but they are fun to share with your close friends.

In a similar way, a marketer can post a great promotion, and share it with a specific circle of people who expressed their wish to receive such promotions. This would not be pushed into everybody’s stream, so it’s not spammy at all. This technique is used more and more by marketers on Google Plus.

Blocking Spammers and Other People

One of the problems on social networks are the spammers. Some people don’t want to be added into circles, because they are more exposed to spamming and unwanted content. They are somewhat right, but not entirely. Some “ingenious marketers“, (read spammers), realized that sharing to circles ensures 100% distribution, whereas sharing to “Public”, doesn’t guarantee any distribution at all. This was exploited to ensure a higher rate of distribution. Well that’s fine, if someone will add you to their circles just to spam you, you can block them, and report them. Google will eventually remove the account if many reports are received for the account. Here is how you block them: Click on the link to their profile, this will bring to to their Google Plus profile page. On the upper left side, under the profile picture, there is a little arrow sign, click on that, and this will give you two options: block and report, or mute. Choose accordingly. I personally report people very rarely, the antispam functions very well on Google+. But I have muted, and blocked maybe 10 people until now. Take a look at the picture below, it’s just a dummy account, that I am using to demonstrate circle management.

Block or Mute A Google Plus Profile

In conclusion, there is no reason to be reluctant to get followed by unknown people, unless you want to duplicate the Facebook on Google+, which is totally doable, but with some extra features.

Why do you need to manage your circles?

Manage Circles To Keep Engaging People Only

The short answer is that you have a limit of people you can follow, and that limit is 5000 people. If you rely on the people you follow to make some noise, and spread your message across, you need engaging, active people. You need to understand who are your engaging followers, which are the inactive people in your circles, and based on that , remove some of the people you follow. Abit more about Circloscope’s features later on the page. For instance, if out of the 5000 people you follow, you get only 2 or three plus ones, or comments, maybe that says something about their engagement with your content. You also have to take in consideration your contribution to their posts. If you never engage in their posts, don’t expect a different treatment from them, but that’s another story, about engaging with your audience.

Manage Follow-Backs

One aspect of of marketing on Google Plus is followers base growth. In other words, the more followers you have the better exposure your content, your posts, and your company will have. Usually, when you follow someone they will follow you back, so potentially, every one of the persons you follow can be a follower. From this perspective you would like to know exactly who followed you back, and who didn’t, and with that knowledge, decide who to keep in your circle and who not, to make place for people who are interested about you or your business. This is a task that you can’t perform without Circloscope.

Circles Relevance

Your circles need to be managed for relevance as well. People you follow need to be in your niche, this is because that gives more relevance to your content, and because they are more likely to engage with content they know about. Don’t think about it as competing with webmasters in your niche, think about it as collaborating to increase the relevancy of everyone. Let’s say your niche is massage therapy, if you get engagement from massage therapists on a post, Google will immediately detect the authority of those engagers, and rank that post, and the linked content, and yourself as an author, or publisher better. So, following the right people is an important aspect of your marketing strategy on Google+. Obviously, this is a tedious task if you do it manually, but with a software tool such as Circloscope, it is so much easier.

 Circloscope – Circle Management Software

Circloscope is a Chrome extension, for Managing Google Plus Circles.

There aren’t many such tools on the market, in fact, I think this is the only tool that allows you to manage Google Plus circles. Circloscope can help you with these tasks and many more:

  • circle growth
  • connect with the right people, (find engagers, and influencers)
  • find event engagers, (targeted audience)
  • clean up circles, (easily find people who didn’t follow you, and find inactive people)
  • many, many more features in a dedicated article.

To get started with Circloscope go to circloscope.com to try their free gversion. For the full version follow this link: Circloscope Premium.

Google Plus for Business – A Must for Any Company on the Web

google+circlesGoogle Plus for Business Beyond Social a Must for Any Business

When Google has launched its new social media platform, everybody thought that this was going to be just another attempt to battle with Facebook and Twitter. Now after a few years since the launch in 2011, every business realizes that Google+ is not just another Facebook, it is way more than that, and not being on Google+ means self sabotage.

Google+ as A Content and Author Confidence Rating Platform

There is a lot happening behind the scenes, that Google won’t disclose, but SEO experts have analyzed all of the patents submitted by Google, and found a lot about their long term plans. They are planning to use their social networking platform not only for content rating, and for getting trust signals, they are also getting indicators of the authority of a web writer or a business. Google+ is a way to identify authors on the WEB, to rate and rank them, and a way to enhance the other online properties with this social layer. In fact, a Google+ profile and business page is a way to increase your website’s ranking, your website’s credibility, and a way to interlink various of your web properties, such as YouTube, website, a business page on Google, etc…

Google+ as A Content Rating and Ranking Platform

Google Plus is also a content rating platform, and this works like any other social media signals. The more vote for a content piece, the better the rankings in SERP. This is a very simplistic way to explain it, but in reality things are more complex. The number of plus ones will give a certain signal to Google, but as Matt Cutts explained, it is just a little piece of the puzzle, and social signals need to confirm every year to be taken in consideration.

Matt Cutts explained that Google’s attempts to use Twitter and Facebook signals were blocked, so they eventually stopped using these signals. However, plus ones will never be blocked, so it will always be a measurable parameter in rankings.

Furthermore, the resharing is probably the most rewarding outcome on Google Plus. A reshare is equivalent with a link to the content shared. Very simple to measure. More reshares, more backlinks.

What is very interesting, is that content liked and reshared will be pushed up in SERP for anyone in the circles involved. So Google+ authority extends onto Google’s search engine. If you get a plus one, or even better a reshare from an authority, someone with one million followers, your content will be displayed higher in SERP for that million followers. From that million followers, all the people who engaged with your post will push further your content to their followers. This is probably one of the best ways to market your content.

Google Plus Authorship and Publisher Tags

Google apparently already started to use its social media platform to get important signals about web content. Obviously, this is still a work in progress, and not all content can be rated on Google +, but most content can be rated. Besides the content quality rating role, Google + also serves as a confidence rating platform. Google authorship, implemented through rel=”author, is one of the ways Google gets its trust signals. The publisher tag is another important trust signal, and ownership verification, and this is specifically meant for businesses. It can complement the authorship tag, and it is regarded as ownership of the web content. If the publisher tag is present, it is assumed that the content belongs to a commercial entity, a business, and the author wrote that piece for the company.

Social Ads – The New Way to Advertise

Maybe it’s worth mentioning here that the advertising giant plans to implement a new way to advertise, and we can call this social advertising. No, don’t even try to compare to what Facebook does with their advertising. In fact Facebook’s advertising is far from what Google plans to do. The way this will work is by pushing ads through social authorities. Social authorities can push ads to their followers, or just ignore them. A poorly conceived advertising will not be pushed by anyone who respects their followers, so there will be a natural selection. If you look at the way Facebook advertises now, it’s just the opposite; they push any ad that brings some money. On the other hand, on Google+ good ads will look like regular content, and users will not be able to distinguish between regular posts and paid ones. Although this looks like something big businesses with serious PR budgets will benefit from, it is also a big opportunity for small businesses.

In conclusion, if you are not already on Google+ you have to be, the game is changing, and the sooner you get your Google+ business page setup, the better for your website.
For a technical person, with some social media experience, this page is a quick start to set up your Google Plus for a Business.