AWS Lake Formation

AWS Lake Formation is a service that makes it easy to set up a secure data lake in days. A data lake is a centralized, curated, and secured repository that stores all your data, both in its original form and prepared for analysis. A data lake enables you to break down data silos and combine different types of analytics to gain insights and guide better business decisions.

However, setting up and managing data lakes today involves a lot of manual, complicated, and time-consuming tasks. This work includes loading data from diverse sources, monitoring those data flows, setting up partitions, turning on encryption and managing keys, defining transformation jobs and monitoring their operation, re-organizing data into a columnar format, configuring access control settings, deduplicating redundant data, matching linked records, granting access to data sets, and auditing access over time.

Creating a data lake with Lake Formation is as simple as defining data sources and what data access and security policies you want to apply. Lake Formation then helps you collect and catalog data from databases and object storage, move the data into your new Amazon S3 data lake, clean and classify your data using machine learning algorithms, and secure access to your sensitive data. Your users can access a centralized data catalog which describes available data sets and their appropriate usage. Your users then leverage these data sets with their choice of analytics and machine learning services, like Amazon Redshift, Amazon Athena, and (in beta) Amazon EMR for Apache Spark. Lake Formation builds on the capabilities available in AWS Glue.

Below are the cmdlets which are available with AWS Lake Formation

CmdletNameServiceOperation
Get-LKFDataLakeSetting                             GetDataLakeSettings                           
Get-LKFEffectivePermissionsForPath                 GetEffectivePermissionsForPath                
Get-LKFPermissionList                              ListPermissions                               
Get-LKFResource                                    DescribeResource                              
Get-LKFResourceList                                ListResources                                 
Grant-LKFPermission                                GrantPermissions                              
Grant-LKFPermissionBatch                           BatchGrantPermissions                         
Register-LKFResource                               RegisterResource                              
Revoke-LKFPermission                               RevokePermissions                             
Revoke-LKFPermissionBatch                          BatchRevokePermissions                        
Unregister-LKFResource                             DeregisterResource                            
Update-LKFResource                                 UpdateResource                                
Write-LKFDataLakeSetting                           PutDataLakeSettings                           

You can also check other AWS Services, and each services cmdlets we are providing.

Click on this Link for an Single place, where you get all the PowerShell cmdlets sorted based on the modules.

You can also refer other blogs on PowerShell at link

You can also refer other blogs on Microsoft at link

And also if you required any technology you want to learn, let us know below we will publish them in our site http://tossolution.com/

Like our page in Facebook and follow us for New technical information.

References are taken from Microsoft and AWS

2 Replies to “AWS Lake Formation”

Leave a Reply

Your email address will not be published. Required fields are marked *