DBA Blogs

The Power of HDFS ACLS

Alejandro Vargas - Wed, 2019-05-01 02:12

Kerberos and Sentry help us defining access privileges for a limited set of object living on HDFS

We can extend the control by using access control lists, ACL's 

Requesting the ACL's on a directory will show us if "dfs.namenode.acls.enabled" is set to tru on HDFS:

[root@linux-x64]# hdfs dfs -getfacl /user
# file: /user
# owner: hdfs
# group: supergroup

getfacl: The ACL operation has been rejected.  Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false.

In this case as it is set to false we need to enable it, for this we go to Cloudera Manager > HDFS > Configuration and search for "dfs.namenode.acls.enabled" and we mark it as enabled.

We need to restart the stale services to enable ACL's

First I'll create a new user for testing

[root@sf_D_DRIVE]# adduser avargas

[root@media]# grep avargas /etc/passwd avargas:x:502:504::/home/avargas:/bin/bash Then I will create a user hdfs a directory inside HDFS for user avargas [root@media]# sudo su -hdfs -bash-4.1$ hdfs dfs -mkdir /user/avargas -bash-4.1$ hdfs dfs -ls /user Found 9 items drwxr-xr-x   - hdfs     supergroup          0 2019-04-29 12:42 /user/avargas drwxr-xr-x   - cloudera cloudera            0 2019-01-24 08:45 /user/cloudera drwxr-xr-x   - mapred   hadoop              0 2018-12-24 23:39 /user/history drwxrwxrwx   - hive     supergroup          0 2017-10-23 09:17 /user/hive drwxrwxrwx   - hue      supergroup          0 2018-12-19 07:44 /user/hue drwxrwxrwx   - oozie    supergroup          0 2017-10-23 09:16 /user/oozie drwxrwxrwx   - root     supergroup          0 2017-10-23 09:16 /user/root drwxr-xr-x   - hdfs     supergroup          0 2017-10-23 09:17 /user/spark As the owner user hdfs I'll assign rwx permissions on the directory /user/avargas to user avargas sh-4.1$ hdfs dfs -setfacl -m user:avargas:rwx /user/avargas -bash-4.1$ hdfs dfs -getfacl /user/avargas # file: /user/avargas # owner: hdfs # group: supergroup user::rwx user:avargas:rwx group::r-x mask::rwx other::r-x Now I'll upload a file as user avargas root@sf_D_DRIVE]# su - avargas [avargas@~]$ hdfs dfs -ls /user > filelist.txt [avargas@~]$ hdfs dfs -put filelist.txt /user/avargas File /user/avargas/filelist is owned by user avargas:supergroup [avargas@~]$ hdfs dfs -ls /user/avargas Found 1 items -rw-r--r--   1 avargas supergroup        680 2019-04-29 13:26 /user/avargas/filelist.txt Directory /user/avargas still owned by hdfs:supergroup, but note the + sign indicating additional permissions assigned on it: [avargas@~]$ hdfs dfs -ls /user | grep avargas drwxrwxr-x+  - hdfs     supergroup          0 2019-04-29 13:26 /user/avargas We can also set ACL’s for several users and groups on a single command

-bash-4.1$ hdfs dfs -setfacl -m user:avargas:rwx,user:hadoop:rwx,group::rwx,other::rwx /user/avargas

-bash-4.1$ hdfs dfs -getfacl /user/avargas
# file: /user/avargas
# owner: hdfs
# group: supergroup
user::rwx
user:avargas:rwx
user:hadoop:rwx
group::rwx
mask::rwx
other::rwx

Note that permissions were not changed for a file

-bash-4.1$ hdfs dfs -ls /user/avargas
Found 1 items
-rw-r--r--   1 avargas supergroup        680 2019-04-29 13:26 /user/avargas/filelist.txt

We can set ACL’s for several users and groups and recursively on all folders and files using the -R flag

-bash-4.1$ hdfs dfs -setfacl -m -R user:avargas:rwx,user:hadoop:rwx,group::rwx,other::rwx /user/avargas

-bash-4.1$ hdfs dfs -ls /user/avargas
Found 1 items
-rw-rwxrwx+  1 avargas supergroup        680 2019-04-29 13:26 /user/avargas/filelist.txt

Categories: DBA Blogs

Rebuilding local non prefix index raises ORA 02149: Specified partition does not exist

Tom Kyte - Tue, 2019-04-30 16:06
I truncated the partition on a table with local partition index. After inserting the records again in the same partition If I try to rebuild the index on that partition it gives ORA 02149: Specified partition does not exist. However i can see the res...
Categories: DBA Blogs

Combine json objects into one document

Tom Kyte - Tue, 2019-04-30 16:06
I need to combine some existing json objects into a new json object. Using the syntax: <code>json3 := json_object('json1' value json1, 'json2' value json2);</code> the existing json objects get wrapped in an extra set of double quotes, and t...
Categories: DBA Blogs

Query Rewrite to improve performance.

Tom Kyte - Tue, 2019-04-30 16:06
Hi, I have this query which takes about 45 minutes to populate the data into AFS_TABLE. The AFS_TABLE is truncated before this insert runs and it does not have any indexes or constraints. The Oracle version is 11.2.0.4. The VW_PAYMENT table is a very...
Categories: DBA Blogs

Writing data to a clob

Tom Kyte - Tue, 2019-04-30 16:06
Hi Tom how to write the table data into clob column in another table in oracle 12.1 version <code>create table abc(sno number(2) not null,file_clob clob); insert into abc(sno) values(1); insert into abc(sno) values(2); insert into abc(sno) va...
Categories: DBA Blogs

Get the most recent 3 transaction records for a customer

Tom Kyte - Tue, 2019-04-30 16:06
I have a customer session table that has session id and session date along with other columns. I want to see the 3 most recent sessions of each customer. Provided the sample SQL for creating data below: <code>CREATE TABLE Customer ( Custo...
Categories: DBA Blogs

Generate Random number and compare it for uniqueness in database

Tom Kyte - Tue, 2019-04-30 16:06
Hi Tom, I have a situation where I have to create a function which will return a unique number (Suffix R and 7 numbers) which will than be compared with data present in database with a particular column which is primary key. if the number generate...
Categories: DBA Blogs

Issue Global temporary table

Tom Kyte - Tue, 2019-04-30 16:06
Hi Tom I have Stored procedure return some data, <code>create PROCEDURE "SP1" (CV_1 IN OUT SYS_REFCURSOR) /*CREATE GLOBAL TEMPORARY table TT_TABL2 ( ORDER_ID NUMBER, REQ_ID NUMBER, TXN_ID NUMBER, ...
Categories: DBA Blogs

Oracle Insert

Tom Kyte - Tue, 2019-04-30 16:06
Hi Tom, How exactly big insert or update getting processed in oracle? Suppose I have to insert >=10GB more records in this table will this much of insert floods the db buffer cache or how it affect the SGA. Is it the same case with update...
Categories: DBA Blogs

problem to drop tables

Tom Kyte - Tue, 2019-04-30 16:06
hi i have a problem when i am trying to drop tables i get this error Error dropping CITIZEN: ORA-04098: trigger 'SYS.DELETE_ENTRIES' is invalid and failed re-validation thanks israel
Categories: DBA Blogs

May 17th AZORA Meetup – Last until September!

Bobby Durrett's DBA Blog - Tue, 2019-04-30 14:28
#meetup_oembed .mu_clearfix:after { visibility: hidden; display: block; font-size: 0; content: " "; clear: both; height: 0; }* html #meetup_oembed .mu_clearfix, *:first-child+html #meetup_oembed .mu_clearfix { zoom: 1; }#meetup_oembed { background:#eee;border:1px solid #ccc;padding:10px;-moz-border-radius:3px;-webkit-border-radius:3px;border-radius:3px;margin:0; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 12px; }#meetup_oembed h3 { font-weight:normal; margin:0 0 10px; padding:0; line-height:26px; font-family:Georgia,Palatino,serif; font-size:24px }#meetup_oembed p { margin: 0 0 10px; padding:0; line-height:16px; }#meetup_oembed img { border:none; margin:0; padding:0; }#meetup_oembed a, #meetup_oembed a:visited, #meetup_oembed a:link { color: #1B76B3; text-decoration: none; cursor: hand; cursor: pointer; }#meetup_oembed a:hover { color: #1B76B3; text-decoration: underline; }#meetup_oembed a.mu_button { font-size:14px; -moz-border-radius:3px;-webkit-border-radius:3px;border-radius:3px;border:2px solid #A7241D;color:white!important;text-decoration:none;background-color: #CA3E47; background-image: -moz-linear-gradient(top, #ca3e47, #a8252e); background-image: -webkit-gradient(linear, left bottom, left top, color-stop(0, #a8252e), color-stop(1, #ca3e47));disvplay:inline-block;padding:5px 10px; }#meetup_oembed a.mu_button:hover { color: #fff!important; text-decoration: none; }#meetup_oembed .photo { width:50px; height:50px; overflow:hidden;background:#ccc;float:left;margin:0 5px 0 0;text-align:center;padding:1px; }#meetup_oembed .photo img { height:50px }#meetup_oembed .number { font-size:18px; }#meetup_oembed .thing { text-transform: uppercase; color: #555; }
AZORA Meetup before summer heats up! Two presentations – Friday, May 17th

Friday, May 17, 2019, 12:00 PM

Republic Services – 2nd Floor Conference Rooms
14400 N 87th St (AZ101 & Raintree) Scottsdale, AZ

8 AZORAS Attending

AZORA Meetup before the summer heats up! We will be featuring two presentations on Friday May 17th meeting. Logistics are the same at friendly Republic Services, an afternoon with other Oracle types between Noon till 4pm. We will follow typical agenda of Lunch – Learn – Learn some more – Let us call it a weekend! Here are the details: Meeting Agend…

Check out this Meetup →

This is our last Meetup until our Summer break. Come check out two great presentations.

Doug Hood from Oracle will talk about the Oracle In-Memory database feature. We appreciate Oracle providing us with this technical content to support AZORA.

AZORA’s own Stephen Andert will be sharing a non-technical presentation on Networking. He just gave the same talk at the national Oracle user group meeting called Collaborate 19 so it will be great to have him share with his local user group.

Looking forward to seeing you there.

Bobby

P.S. AZORA is the Arizona Oracle User Group, and we meet in the Phoenix, Arizona area.

Categories: DBA Blogs

OGG-01298 could not find column TRANSACTION

VitalSoftTech - Mon, 2019-04-29 19:39
Question: When upgrading GoldenGate from 11 to 12c, on starting up the Replicat I get the following error. What caused this and how is it resolved? OGG-01298 Column function diagnostic message: could not find column “TRANSACTION”. Answer: The OGG-01298 error is returned on the Replicat startup after the GoldenGate is upgraded to 12c. This happens […]
Categories: DBA Blogs

.NET Core Connection String to RAC database

Tom Kyte - Mon, 2019-04-29 02:46
Hello, Ask Tom Team. We have a .Net Core app connecting to a Oracle 18c RAC database. We are not using tnsnames.ora file (there is no oracle client installed on the client-server side). We are using managed .Net Core oracle client and the followin...
Categories: DBA Blogs

Stored Procedure behaves randomly - sometimes takes 15 minutes where it should take 15 milli-seconds.

Tom Kyte - Mon, 2019-04-29 02:46
We developed a procedure to be used in database-A and in database-A. In this procedure we send an input parameter on the basis of which a select statement searches the data from a table located in remote database-B. The selected columns are set in Ou...
Categories: DBA Blogs

Cluster Waits Rac database

Tom Kyte - Mon, 2019-04-29 02:46
Hello, Ask Tom Team. I have a two-nodes RAC database running 18.4.0 and I'm running a load test (lots of inserts). Basically, two tables are being hit. These tables are using identity column and have a relationship (fk). I'm checking OEM 13c and t...
Categories: DBA Blogs

High commit wait on RAC database

Tom Kyte - Mon, 2019-04-29 02:46
Hello, Ask Tom Team. I have a two-nodes RAC database running 18.4.0 and I'm running a load test (lots of inserts). Basically, two tables are being hit. These tables are using identity column and have a relationship (fk). I'm checking OEM 13c and t...
Categories: DBA Blogs

Redo log file size and Database Performance

Tom Kyte - Mon, 2019-04-29 02:46
Hi there, We have a database setup as follows 1) SQL server -- Where transactions are happening here 2) Oracle(DSS)-- we are transferring the incremental data from SQL server(NOT ALL THE TABLES ONLY 22 TABLES) with the help of a 24/7 running ...
Categories: DBA Blogs

Redo logs Sizing

Tom Kyte - Mon, 2019-04-29 02:46
Hello, Ask Tom Team. What is the best practice: small redo log groups or have a few but with a bigger size? Thanks in advanced. Regards,
Categories: DBA Blogs

RMAN - Full vs. Incremental - performance problem

Tom Kyte - Thu, 2019-04-25 22:46
Hi. I am testing RMAN. I have run an incremental 0 and an incremental 1 cumulative test without having any database activity in-between these test. I am using OmniBack media manager with RMAN. I have three databases. Here are the timing results ...
Categories: DBA Blogs

Subtract time from a constant time

Tom Kyte - Wed, 2019-04-24 10:06
Hello there, I want to create a trigger that will insert a Time difference value into a table Example: I have attendance table Sign_in date; Sign_out date; Late_in number; Early_out number; Now I want to create a trigger that will insert la...
Categories: DBA Blogs

Pages

Subscribe to Oracle FAQ aggregator - DBA Blogs