Scenario 四:Use access keys to complete data flow for the same account
Use the RAM user to create a data transformation job 时,您可以通过访问密钥完成同账号内的日志数据流转。
Step 1: Use Alibaba Cloud Account 1 to obtain the AccessKey pair of RAM User S
- Log on to the [RAM console] by using an Alibaba Cloud account.(https://ram.console.aliyun.com/overview)。
- Obtain the AccessKey pair of RAM User S. For more information, see Create an AccessKey pair for a RAM user.
Note The AccessKey secret of a RAM user is displayed only when you create the AccessKey pair for the RAM user. You cannot query the AccessKey secret after the AccessKey pair is created. Keep your AccessKey secret confidential. If the AccessKey pair of a RAM user is disclosed or lost, you must create another AccessKey pair. You can create a maximum of two AccessKey pairs for a RAM user. |
Step 2: Use Alibaba Cloud Account 2 to obtain the AccessKey pair of RAM User T
Log on to the [RAM console] by using an Alibaba Cloud account.(https://ram.console.aliyun.com/overview)。
Create a custom policy on the JSON tab.The policy grants the permissions to read data from the source Logstore.In this example, create a policy named ori_read. For more information, see Create custom policies.The following table describes the key parameters:
|Key parameter| Note | | -- | -- | | Name | Enter a name for the custom policy.Example:**ori_read** | | Policy content | Replace the content in the code editor with the following script.<br>For example, the name of the source project is log-project-prod. The name of the source Logstore is access_log_output.You can replace the project and Logstore names based on your business requirements.<pre>{<br> "Version": "1",<br> "Statement": [<br> {<br> "Action": [<br> "log:ListShards",<br> "log:GetCursorOrData",<br> "log:GetConsumerGroupCheckPoint",<br> "log:UpdateConsumerGroup",<br> "log:ConsumerGroupHeartBeat",<br> "log:ConsumerGroupUpdateCheckPoint",<br> "log:ListConsumerGroup",<br> "log:CreateConsumerGroup",<br> ],<br> "Resource": [<br> "acs:log:*:*:project/log-project-prod/logstore/access_log",<br> "acs:log:*:*:project/log-project-prod/logstore/access_log/*",<br> ],<br> "Effect": "Allow"<br> }<br> ]<br>}</pre> |
Grant the read permissions on the source Logstore to RAM User S.。 For more information, see Grant permissions to a RAM role.The following table describes the key parameters.
| Key parameter| Note | | -------| --------- | | **Authorized Scope** |Select **Alibaba Cloud Account**.The permissions granted to the RAM user take effect on resources within the current Alibaba Cloud account. | | **Principal** | Select**UserS**。 | | **Custom Policy** |Select**ori_read**。 |
Step 3:Use Alibaba Cloud Account to grant the write permissions on the destination Logstores to RAM User T.
Log on to the [RAM console] by using an Alibaba Cloud account.(https://ram.console.aliyun.com/overview)。
Create a custom policy on the JSON tab.The policy grants the permissions to write data to the destination Logstores.例 In this example, create a policy namedwrite。 For more information, see Create custom policies.The following table describes the key parameters.
|Key parameter| Note | | -- | -- | |Name| Enter a name for the custom policy.Example:**write** | | Policy content | Replace the content in the code editor with the following script.<br>For example, the name of the source project is log-project-prod. The name of the source Logstore is access_log_output.You can replace the project and Logstore names based on your business requirements.<br><pre>{<br> "Version": "1", <br> "Statement": [ <br> { <br> "Action": [ <br> "log:Post*",<br> "log:BatchPost*" <br> ],<br> "Resource": "acs:log:*:*:project/log-project-prod/logstore/access_log_output",<br> "Effect": "Allow" <br> }<br> ]<br>} </pre> |
Grant the write permissions on the destination Logstore to RAM User T. For more information, see Grant permissions to a RAM role.The following table describes the key parameters.
Key parameter Note Authorized Scope Select Alibaba Cloud Account.The permissions granted to the RAM user take effect on resources within the current Alibaba Cloud account. Principal Select User T Custom Policy Selectwrite。
Step 4:Use the RAM user to create a data transformation job
Log on to the Simple Log Service console by using the RAM user.
Go to the data transformation page. a. In the Projects section, click the desired project. b. In the left-side navigation pane, click Log Storage. On the Logstores page, click the desired Logstore. c. On the query and analysis page, click Data Transformation.
In the upper-right corner of the page, specify a time range for the required log data. Make sure that log data exists on the Raw Logs tab.
In the code editor, enter the following data transformation statement. For more information, see Data processing syntax.(https://www.alibabacloud.com/help/en/doc-detail/125439.htm?spm=a2c4g.11186623.0.0.10b9708cbP33kd#concept-1130584)。
Preview data in advanced mode. a. Select Quick. You can select Quick or Advanced.For more information, see Preview mode overview. b. Click Preview Data. View the transformation results.
- If the data fails to be transformed because the syntax of the transformation statement or the permissions are invalid, troubleshoot the failure as prompted.
- If the transformed data is returned as expected, go to the next step.
Create a data transformation job. a. Click Save as Transformation Job. b.In the Create Data Transformation Job panel, configure the parameters and click OK. For more information about parameter configurations, see Get started with data transformation.The following table describes the key parameters.
Key parameter Note Authorization Method SelectAccessKey AccessKey ID RAM 用户 S 的 AccessKey ID AccessKey Secret RAM 用户 S 的 AccessKey Secret Authorization Method in Storage Destination SelectAccessKey AccessKey ID RAM 用户 T 的 AccessKey ID AccessKey Secret RAM 用户 T 的 AccessKey Secret
After the data transformation job is created and run, data can be transferred across the two Alibaba Cloud accounts.For more information, see Manage a data transformation job.