If you are looking to transfer your data from on-premise NAS shares to S3, and want to use native NFS/SMB protocols for the data transfer to S3, you can use AWS File Gateway. You can also mount the NFS/SMB shares as path/drives on your backup applications (like Veeam etc) to take backup of on-premise data to AWS Cloud on S3.
You would be able to deploy the AWS File Gateway on a VM which uses VMware ESXi or Hyper-V Hypervisor on on-prem. However, if you want to play out and test the features of AWS File Gateway, you can launch a VM on VMware workstation player and perform some basic functionality testing with less compute,memory resources than recommended. Here, I will show you steps to deploy the ova template on to a VM on VMware workstation player.
Login to AWS web console and go to storage gateway and chose Create Gateway, then you would see the below screen which would ask you to chose the gateway type. Select the file Gateway and click Next.
Then, chose VMware ESXi and download the ova image.
After Downloading the ova image, unzip the AWS-Appliance directory and launch VMware workstation. Deploy OVA template choosing option Open a Virtual Machine.
Navigate to the path and chose the .ova file where you have downloaded.
Provide the name for the new VM and storage location for the VM.
After the ova file completed importing, create a new hard disk of 150 GB for cache disk storage. You might be able to create a smaller one and assign to the VM.
After creating the disk, power on the VM and login to the local console. You can use admin/password for the login credentials.
Once you logged in, you would see the following options. I let the DHCP to dynamically assign the IP address for the VM . If you want to configure static IP you can do it by choosing option 2.
Now, you have to ensure that the VM has network connectivity to AWS Storage gateway endpoints running option 3: Test Network Connectivity. You would want to have the network checks passed to have the gateway activated. Once you chose option 3, you would need to chose the option 1:Storage Gateway
Then, chose the Public endpoint as we would be activating the gateway into a Public Endpoint.
Select the region number in which you would be activating the gateway, I chosen 17 as I would be activating the gateway in that region. My networks checks return passed.
Once the network checks passed, you can activate the gateway from the AWS storage gateway web console providing the IP address of the gateway VM. Chose File Gateway and chose the host platform as VMware ESXi and click Next:
Chose Public as endpoint type click next and provide the IP address of the gateway VM IP and chose connect to gateway. Here the local system where you are activating the gateway on browser, should be able to connect on port 80 to the VM IP.
If you are able to successfully connect to gateway, you would see this page to Activate gateway. If you are unable to connect to gateway, check telnet<gateway IP address> 80. Select Activate Gateway.
In this phase, gateway is trying to recognize the local cache disk and takes couple of minutes.
Once the gateway recognizes the local disk for cache, you would see this page. I am skipping the configure logging part. But, if you want you can configure logging with a CloudWatch log group and get notified about S3 errors in CloudWatch.
Once you click save and continue with default disable logging, you would see the gateway is activated and created successfully.
After the gateway is created, you can create a SMB or NFS file share and test upload a sample file to the client and see that file transferred to S3. In my case, I enabled SMB guest access with guess password and then created a SMB share with guest access. For enabling Guest Access, go to Actions and Chose Edit SMB settings and set Guest password.
Create File share, chose an S3 bucket which already exists in your account, select guest access and create the file share. You can leave the defaults for IAM role and other values and select create file share.
Once the file share is created, you can mount the file share using the following command from Windows command prompt.
net use [WindowsDriveLetter]: \\192.168.157.128\aws-filegateway /user:sgw-3BD93D52\smbguest
You would be able to see the mapped drive on Windows explorer and upload a sample test file. You would see that file uploaded to your S3 bucket of the file share.
So the above file which is created on my SMB client is uploaded to S3 bucket. You can also create NFS share and export it to your Linux clients. You can also join the gateway to AD domain and control access to the shares/folders using NTFS ACLs.
Nice article! Very useful!