Hello all -

I am playing with Drill (I have 1.2 running right now) and I am trying to
figure out a way to do some logging in a logical way.  I know I am running
outside the norm, in that I am running my drill bits using marathon on
mesos, but for a moment (unless it's THAT that is breaking my logging :)
ignore that fact :)

I am using MapR, which is nice because it lets me NFS mount my distributed
file system (MapRFS) Thus, when I run a drillbit, I package up the
drill-1.2.0 directory (I am using the MapR packaged version) and then in
Marathon I create a app profile like this

{
"cmd": "./drill-1.2.0/bin/runbit --config
/mapr/zetapoc/mesos/prod/drill/conf.std",
"cpus": 12.0,
"mem": 22528,
"id": "zetadrill",
"instances": 5,
"uris":["hdfs:///mesos/prod/drill/drill-1.2.0.tgz"],
"constraints": [["hostname", "UNIQUE"]]
}

Pretty basic, for non Mesos folks, it takes the URI (the tgz) downloads it
to the Mesos Sandbox, untars it there, and runs 'runbit' using cgroups.
Thus the memory and cpu specifications.  I am passing it a full path to the
config location which is just the /conf  copied out of the drill-1.2.0
directory and modified. Thus, all bits are pulling from the same conf. (I
could package the conf in the tgz, this makes its easy to change a config
and not have to repackage the tgz)

Also, as you can see I am using the constraint to only have one per node,
this is handy from a port conflict point of view.

Ok, that out of the way, when I run it, in my drill-env.sh, I am setting:
export DRILL_LOG_DIR="./drill-1.2.0/log"

What I "think" that means is it should log inside the sandbox. And sure
enough, when I look in the sandbox in mesos, in the log directory, there is
a profiles folder and the sys.drill (query logs... I think?)

Great.

Now, in the stdout on each node that started drill, I do have the error
below... and additionally, I don't see the drillbit.log and drillbit.out
files.  (Is that because of the errors below?) Wouldn't these be in
./drill-1.2.0/log as well?

Side note: If I am doing a CTAS to Parquet, one of my nodes starts spitting
out INFO logs to the console like crazy during the write, not sure what
that is all about...

Ok, back to logging. I am trying to organize this, ideally, I want to
understand logging deeply to meet a number of requirements

1.  I'd like all profiles/sql audit logs logged to one place.  Ideally,
this would make it so if I click profiles on any node, I'd see the
historical queries for all nodes, not just the one I am connected to.  Is
this possible? Right now, I am only seeing the historical queries if I am
connected to a node that acted as foreman for that query

2.  I'd like to keep all my logs in MapRFS with NFS, thus I'd like all
nodes, for logfiles that have the same names (like drillbit.log and
drillbit.out) to write to a file in the same networked location, but have
the hostname in the logfile name.  This would allow me to log to the same
place on all nodes, but I'd need to know how to do the hostname thing (if
that is possible)

3. Are there any gotchas to what I am trying to do?  What about having more
control over pruning? i.e. Can I keep the sql audit logs in a directory,
and those will show up in the console, but have an job or setting (inside
or outside drill) to move them to a "final" resting place for longterm
auditing?

I know some of these things may be obvious to users of drill, I have been
having a hard time digging through the docs (for example, I am not sure
where to find a good explanation of ENV vars that may help me here), any
advice would be welcome!

John

Error on in STDOUT on sandbox.. is this hurting me?
16:35:22,996 |-INFO in
ch.qos.logback.core.rolling.RollingFileAppender[QUERY] - Active log file
name:

16:35:22,996 |-INFO in
ch.qos.logback.core.rolling.RollingFileAppender[QUERY] - File property is
set to []

16:35:22,998 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[QUERY] - openFile(,true)
call failed. java.io.FileNotFoundException:  (No such file or directory)

        at java.io.FileNotFoundException:  (No such file or directory)

        at      at java.io.FileOutputStream.open(Native Method)

        at      at
java.io.FileOutputStream.<init>(FileOutputStream.java:221)

        at      at
ch.qos.logback.core.recovery.ResilientFileOutputStream.<init>(ResilientFileOutputStream.java:28)

        at      at
ch.qos.logback.core.FileAppender.openFile(FileAppender.java:149)

        at      at
ch.qos.logback.core.FileAppender.start(FileAppender.java:108)

        at      at
ch.qos.logback.core.rolling.RollingFileAppender.start(RollingFileAppender.java:86)

        at      at
ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:96)

        at      at
ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:317)

        at      at
ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:196)

        at      at
ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:182)

        at      at
ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)

        at      at
ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:149)

        at      at
ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:135)

        at      at
ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:99)

        at      at
ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:49)

        at      at
ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)

        at      at
ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:148)

        at      at
org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:85)

        at      at
org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55)

        at      at org.slf4j.LoggerFactory.bind(LoggerFactory.java:1