Evaluating a calculated field on the server

7 messages Options
Embed this post
Permalink
Jake Traynham

Evaluating a calculated field on the server

Reply Threaded More More options
Print post
Permalink
Hello all,

   I know we've discussed this before, but I don't completely remember
the answer and I'm not sure if the answer has changed with recent
versions of server.  So, here goes:

   Is there any known way to make a calculated field evaluate on the
server side?  I know it can be done with a schema change to the
database, but I'm looking for a way that can be done without opening
Define Database and modifying something.  Has anyone figured out a way
to do this?  Maybe with some ExecuteSQL trickery?  (Like, if the schema
is changed with a call to ExecuteSQL, does the server re-evaluate
everything, or does the client?)

   The reason I'm asking is because my brother and I are discussing
options for modifying server plug-in preferences remotely.  It would be
really nice if we could just have our customers throw a database on the
server and adjust settings in it that set the preferences on the server.
  However, the only way that would work is if I could actually get the
settings database to evaluate on the server, thus talking to the server
plug-in to save the settings on the server.  Any ideas?

   An alternate solution would be to somehow trigger a scheduled script
on the server.  Is that possible?

Thanks,
Jake

--
Jake Traynham
Owner, CNS Plug-ins
http://www.cnsplug-ins.com/
Jesse Barnum-2

Re: Evaluating a calculated field on the server

Reply Threaded More More options
Print post
Permalink
Hey Jake - I don't know of any way to get plugins to evaluate calcs  
on the server, except for schema changes like you mention.

I think if I were trying to do what you're talking about, I would  
implement a little mini HTTP server in my plugin. This is exactly  
what we do with AdminAnywhere - we actually embed our own copy of  
Tomcat into AdminAnywhere.

You can trigger a scheduled script on the server using the fmsadmin  
command line feature - again, this is how AdminAnywhere works. If you  
want any help interfacing with the fmsadmin app, I've written a nice  
object-oriented Java wrapper around it - don't know if that would be  
any help or not.

--Jesse Barnum, President, 360Works
http://www.360works.com 770-234-9293
FBA Platinum, FileMaker 8 Certified
== SuperContainer: A Better Container Field ==

On Jun 24, 2008, at 4:56 PM, Jake Traynham wrote:

> Hello all,
>
>    I know we've discussed this before, but I don't completely remember
> the answer and I'm not sure if the answer has changed with recent
> versions of server.  So, here goes:
>
>    Is there any known way to make a calculated field evaluate on the
> server side?  I know it can be done with a schema change to the
> database, but I'm looking for a way that can be done without opening
> Define Database and modifying something.  Has anyone figured out a way
> to do this?  Maybe with some ExecuteSQL trickery?  (Like, if the  
> schema
> is changed with a call to ExecuteSQL, does the server re-evaluate
> everything, or does the client?)
>
>    The reason I'm asking is because my brother and I are discussing
> options for modifying server plug-in preferences remotely.  It  
> would be
> really nice if we could just have our customers throw a database on  
> the
> server and adjust settings in it that set the preferences on the  
> server.
>   However, the only way that would work is if I could actually get the
> settings database to evaluate on the server, thus talking to the  
> server
> plug-in to save the settings on the server.  Any ideas?
>
>    An alternate solution would be to somehow trigger a scheduled  
> script
> on the server.  Is that possible?
>
> Thanks,
> Jake
>
> --
> Jake Traynham
> Owner, CNS Plug-ins
> http://www.cnsplug-ins.com/
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Kent Lendrum

Re: Evaluating a calculated field on the server

Reply Threaded More More options
Print post
Permalink
In reply to this post by Jake Traynham
Hi Jake,

I had similar questions of Server Side Plugins, and have included the  
replies I received from FM.

As I understand it, the only time server-side plugins are called are
- Schema Updates
- Finds/Queries
- Server Side Scripts


Therefore, I would suggest you use a 'Server Side Script' to configure  
your plugins.

I currently use my own plugin on the server, but also use Troi File  
plugin and Dacon's mail.it plugin

For the later two,  i pass the registration information each time I  
run the server side script


The following is feedback from FileMaker when I queried about server  
side plugins.

Kent.




> A plugin for a calc is only called on FileMaekr server when:
>
> * During schema updates a stored calc containing an external  
> function needs
> to be reevaluated.
> * During a find request on an unstored calculation field that  
> depends on an
> external function.
> * During the execution of server side scripts.
>
> For web clients, the FM Web Publishing process has its own set of  
> plug-ins
> that are loaded and they are executed during web requests.
>
> Note that plug-ins need to worry about being called on different
> threads/processors at the same time so anything that is shared between
> threads (like globals) need to be protected by some form of  
> synchronization
> support. This is more important when the plug-in is loaded into the
> FileMaker server process or the FM Web Publishing process when  
> multiple
> sessions (for server side scripts or web requests) are executing at  
> the same
> time.



>> in the Plugin API,  there is the  kMayEvaluateOnServer flag.
>> - I have set this for functions which take some data, and return a  
>> result.
>> - I have NOT set this for functions which either store or return  
>> data that I
>> am storing in globals between function calls.
>
> That sounds like a safe plan for FileMaker Pro/Advance.
>
>> If I had a function, which does use global storage,  I could also  
>> set this to
>> evaluate on the server  (for use in 'Server Side Scripts' say),  
>> provided I am
>> aware of the potential for contamination of another server side  
>> script.
>>
> The only FileMaker
> code that looks at that flag is the query optimizer when it  
> determines what
> hosts will do what parts of the work for a find request. If that  
> flag is on
> then it will not send a query to a remote host if it would require  
> executing
> that external function on a remote host, instead the query will be  
> done on
> the client.
>
>> Does the Server Plugin also call the 'Do_PluginIdle'  API function ?
>>
> Server side plug-in do get idle calls but I don't think they ever  
> get any
> "safe" idle calls. You can find out what you are being called under by
> getting the version string or looking at the FMX_Application value  
> that you
> get during the kFMXT_Init call.
>
> As for globals, they are shared between all the sessions that are  
> running in
> that process. So on the server and on the web publishing engine  
> you'll need
> some thread protection on any globals or use thread specific  
> globals. Note
> also that a specific session may not stay on the same thread between  
> script
> instantiations or web requests so it would not be safe to use thread
> specific globals for any type of long term storage.


--------------------------------------------------------------------
I was always taught to respect my elders, but it keeps getting harder  
to find one.

On 25/06/2008, at 8:56 AM, Jake Traynham wrote:

> Hello all,
>
>   I know we've discussed this before, but I don't completely remember
> the answer and I'm not sure if the answer has changed with recent
> versions of server.  So, here goes:
>
>   Is there any known way to make a calculated field evaluate on the
> server side?  I know it can be done with a schema change to the
> database, but I'm looking for a way that can be done without opening
> Define Database and modifying something.  Has anyone figured out a way
> to do this?  Maybe with some ExecuteSQL trickery?  (Like, if the  
> schema
> is changed with a call to ExecuteSQL, does the server re-evaluate
> everything, or does the client?)
>
>   The reason I'm asking is because my brother and I are discussing
> options for modifying server plug-in preferences remotely.  It would  
> be
> really nice if we could just have our customers throw a database on  
> the
> server and adjust settings in it that set the preferences on the  
> server.
>  However, the only way that would work is if I could actually get the
> settings database to evaluate on the server, thus talking to the  
> server
> plug-in to save the settings on the server.  Any ideas?
>
>   An alternate solution would be to somehow trigger a scheduled script
> on the server.  Is that possible?
>
> Thanks,
> Jake
>
> --
> Jake Traynham
> Owner, CNS Plug-ins
> http://www.cnsplug-ins.com/
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Jake Traynham

Re: Evaluating a calculated field on the server

Reply Threaded More More options
Print post
Permalink
In reply to this post by Jesse Barnum-2
Hiya,

>
> I think if I were trying to do what you're talking about, I would
> implement a little mini HTTP server in my plugin. This is exactly
> what we do with AdminAnywhere - we actually embed our own copy of
> Tomcat into AdminAnywhere.
>

   Yea, this is the other option we are looking at .. just having the
plug-in listen on a port for commands.  Not necessarily an http based
protocol, but something.  The only problems I was seeing with this is if
someone has a firewall on the server computer, they'll have to open up
some ports ... and if they have more than one of our plug-ins installed,
they'll be a different port for each plug-in.  This option is probably
our fallback option anyway, but we were just brainstorming other ideas.

Thanks,
Jake



--
Jake Traynham
Owner, CNS Plug-ins
http://www.cnsplug-ins.com/
Jake Traynham

Re: Evaluating a calculated field on the server

Reply Threaded More More options
Print post
Permalink
In reply to this post by Kent Lendrum
Hi Kent,

>> A plugin for a calc is only called on FileMaekr server when:
>>
>> * During schema updates a stored calc containing an external function
>> needs
>> to be reevaluated.
>> * During a find request on an unstored calculation field that depends
>> on an
>> external function.
>> * During the execution of server side scripts.

   Hmmm ... that second one sounds odd, but might could be leveraged if
I understand it.  Is that saying that if a table had an unstored calc
field that references a plug-in and you perform a find looking for a
value in that unstored calc field, that it would be evaluated on the
server if every plug-in function inside it was able to be evaluated on
the server?  If so, it seems like you could script something like that
and the side-effect of evaluating the unstored calculation could do
whatever you wanted to with a server side plug-in.....

Jake



--
Jake Traynham
Owner, CNS Plug-ins
http://www.cnsplug-ins.com/
Kent Lendrum

Re: Evaluating a calculated field on the server

Reply Threaded More More options
Print post
Permalink
Hi Jake,

What I'm not clear on, it the interaction between a server plugin,  
within the 'find' context,  v's a server side script.

When my 'Server Side Scripts' run,  they are shown as another client  
connection  (ie. they show in Server Admin as a connected client)

This leads me to believe that a 'Server Side Script' is another  
instance,  so does this mean the plugin is loaded again ?


I haven't tested this myself.  A test would be to load a 'global'  
variable with something and see if it returns in another instance ???


But, yes.  I don't see any reason why you couldn't get it to evaluate  
as part of a 'find' request :)


I've used both the File & mail plugins successfully as part of 'Server  
Side Scripts',  but haven't tested the 'Find' aspects.


Kent.




On 25/06/2008, at 9:36 AM, Jake Traynham wrote:

> Hi Kent,
>
>>> A plugin for a calc is only called on FileMaekr server when:
>>>
>>> * During schema updates a stored calc containing an external  
>>> function
>>> needs
>>> to be reevaluated.
>>> * During a find request on an unstored calculation field that  
>>> depends
>>> on an
>>> external function.
>>> * During the execution of server side scripts.
>
>   Hmmm ... that second one sounds odd, but might could be leveraged if
> I understand it.  Is that saying that if a table had an unstored calc
> field that references a plug-in and you perform a find looking for a
> value in that unstored calc field, that it would be evaluated on the
> server if every plug-in function inside it was able to be evaluated on
> the server?  If so, it seems like you could script something like that
> and the side-effect of evaluating the unstored calculation could do
> whatever you wanted to with a server side plug-in.....
>
> Jake
>
>
>
> --
> Jake Traynham
> Owner, CNS Plug-ins
> http://www.cnsplug-ins.com/
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Jake Traynham

Re: Evaluating a calculated field on the server

Reply Threaded More More options
Print post
Permalink
Hello all,

   Okay, based on the information that Kent sent, I've done all kinds of
tests and here are my conclusions:

   As we know, there are 3 ways that a calculation will use a plug-in on
the server as opposed to the client.  Each of the 3 ways is listed below
along with my findings for each:

1. During the execution of server-side ScriptMaker scripts.
   -- This should be obvious.  FileMaker Server allows you to create a
Schedule that can call any ScriptMaker script in any file on the server.
  That script can make any calls to server side plug-ins.  This,
however, is only controlled server side.  There is no way a client can
tell the server to run a schedule (without external help anyway).

2. During schema updates a stored calculation containing an external
function to be reevaluated.
   -- The key word here is "reevaluated", which basically boils down to
only schema changes to the actual stored calculation in question will
cause it to be reevaluated on the server.  Adding or removing fields or
changing any of the schema not related to this stored calculation will
not cause it to reevaluate on the server.  Only changing the actual
stored calculation in a significant way (ie, you can't just add a
comment) will cause it to reevaluate on the server.
   -- If you make the stored calculation a global, it only appears to
evaluate on the server the very first time (ie, right after you create
it).  I was not able to get it to reevaluate on the server ever again no
matter how I changed the calculation.  Note though that closing and
reopening the file will appear to make it evaluate on the server, but
it's simply giving you the original value it calculated; it doesn't
actually reevaluate it.  If you think about how global fields work with
served databases, this should make sense.
   -- If the stored calc relies on a secondary field (meaning if you
update that secondary field, the stored calc will reevaluate) and you
have a client-side plug-in use the ExecuteSQL API function to set that
secondary field, the stored calc will reevaluate on the client, not the
server.  It seems this should be obvious, but I've included it for
completeness sake because it is one of the things I tested.

3. During a find request on an unstored calculation field that depends
on an external function.
   -- If you create an unstored calculation field in your database and
then go into Find Mode and type something to find in that unstored
calculation field, it will indeed evaluate this calculation on the
server if it can.  It determines if it can evaluate the calculation on
the server based on the kMayEvaluateOnServer flag of an external
function.  Note that this flag must be defined on both the client-side
and server-side plug-in for all the external functions used in the
calculation.  Otherwise, you will get inconsistent results and it may
even cause the server or the admin to crash (I was having some really
weird crashes testing this stuff .. see the P.S.).
   -- Since a manual find works, this "feature" can be scripted with a
Perform Find script step.  It can also be triggered from a button using
the Perform Find button step.
   -- Doing a "find" from a client-side plug-in using the ExecuteSQL API
function (ie, a "select") does not cause the calculation to evaluate on
the server.  The client appears to handle all queries done through the
ExecuteSQL API function.  This is unfortunate.
   -- There's an interesting side-effect of doing this: If your unstored
calculation uses a plug-in function that returns different results
depending on if it's evaluated on the client or the server, then the
found records returned to you may not appear to match what you were
searching for.  To put this in example form, I was testing with the
Version function of my plug-in.  On the client side, the version
function returns something like "Plugin Name v.1.0".  On the server
side, the version function returns something like "Plugin Name SE
v.1.0".  So, if I do a find for "SE" in this unstored calc, the server
evaluates the Version function on the server, sees the "SE" in the
result, and matches the record.  But, after performing the find, the
client side shows me the resulting records and reevaluates the unstored
calc on the client resulting in a value that does not contain "SE".  So,
it appears to have found records that don't match what I searched for.
I'm still pondering this in my head if it can be exploited in some
interesting way.

   So, of the 3 ways mentioned above, the last method of performing a
find in an unstored calculation is actually a useful and exploitable way
of forcing a calculation to evaluate on the server and thus interact
with a server-side plug-in from a client.  I'll leave it up to the
reader to figure out how to get information from the server-side plug-in
to be returned to the client. :)

   Thanks for the kick-off in the right direction, Kent.  And I hope
everyone else finds this information useful.

Jake

P.S. Here's some interesting things I found out while I was testing this:
   - It appears that if you make a stored calculation that evaluates
FieldNames(Get(FileName);Get(LayoutName)), the server will go bye-bye.
Fun, eh?  Perhaps I should report this. :)
   - If the client-side plug-in does not define kMayEvaluateOnServer for
an external function, but the server-side plug-in does, it seems to
randomly pick which plug-in to ask about whether or not the function can
evaluate on the server or not.  It seemed inconsistent anyway; perhaps
there was some obscure pattern I wasn't seeing.  Also, when I was
testing all this and realizing that I didn't have the
kMayEvaluateOnServer flag set, the server admin java app kept crashing!
  Not the database server itself, but the admin.  It appeared that maybe
the database server was misbehaving though in that the names and
descriptions of the plug-ins in the admin would sometimes disappear or
have random characters in them.  Perhaps the admin was having some
buffer overflow issues based on bad data from the server?  I dunno.

--
Jake Traynham
Owner, CNS Plug-ins
http://www.cnsplug-ins.com/