Genii Weblog


Civility in critiquing the ideas of others is no vice. Rudeness in defending your own ideas is no virtue.


Thu 28 Jan 2021, 12:58 PM
Inline PNG image
Today, Genii Software releases version 4.60 of CoexLinks Migrate, our premier mail migration tool for Domino email databases and archives. Along with a wrap up of various fixes for specialized rendering situations required by individual customers, this version features better control of email layout regions, multibyte character sets in group names, and a performance gain of approximately 20% over the previous version.
 
You only get once change to migrate your data. Do it right.
 
Request an evaluation license today, and see for yourself.
 

Copyright © 2021 Genii Software Ltd.

Tags:

Mon 25 Jan 2021, 11:28 AM
Inline JPEG image
 
Continuing on from last week's post, we are looking at better ways to retrieve data based on two factors:
 
1) How much data do you need from any given record, and which records do you need it from? If there is related data in another record, can you get it without pulling in loads of stuff you don't need? 
 
2) What format should the data be in? How should it be structured or grouped for maximum usability? While you can certainly write code in your application to reformat or restructure, that adds complexity and room for error on your client.
 
Again using Exciton Boost 4.6 with the Curb Appeal database, we'll examine a couple of additional Fancy Tricks (aka "useful features").
 
 
Fancy trick 3: - Saving preliminary values
This may not sound so fancy, but sometimes the value you want is the result of multiple operations. Rather than passing back all the pieces and doing the processing on the client, which might require multiple requests, we can save the results of a formula or rich text operation into a temporary field, and then use those temporary fields to create the result.
 
On Friday, I posed a challenge called Sorted lists of bullets. In the real estate listings in Curb Appeal, there are usually two, occasionally, three columns of bullet items under the Features heading. These are rich text fields, but all we really care about are the text values for each bullet. In our second Fancy Trick from the first post, I talked about using rich text items as a data source.
 
Inline JPEG image
 
Using that trick, we could get the list of bullets from each of the three rich text fields, Col1, Col2, and Col3. (Imaginative naming, eh?) For this fairly simple example, we could return all three lists as separate arrays, then merge the arrays, then sort, all using JavaScript or whatever language. The item list to retrieve these would look something like this:
 
Col1[BulletList *;$UnformattedText]{A}=bullets1,Col2[BulletList *;$UnformattedText]{A}=bullets2,Col3[BulletList *;$UnformattedText]{A}=bullets3 
 
This would produce three arrays named bullets1, bullets2, and bullets3. But this is where the Fancy Trick comes in. By changing the type string in curly braces from {A} to {AP},  with the P meaning Preliminary, none of the three arrays would appear in the result. Instead, a multi-value temporary text list item would be be added. It uses the name preceded by two dollar signs, e.g., $$bullets1 plus a period plus the chunk property, so the field $$bullets1.UnformattedText is now a Notes text list.
 
Col1[BulletList *;$UnformattedText]{AP}=bullets1,Col2[BulletList *;$UnformattedText]{AP}=bullets2,Col3[BulletList *;$UnformattedText]{AP}=bullets3 
 
As a final step, we can use formula language now that we have simple text list fields, so we add an additional item as 
 
@Sort($$bullets1.UnformattedText:$$bullets2.UnformattedText:$$bullets3.UnformattedText){A}=bulletsSorted 
 
The three text lists get appended together then sorted with @Sort, and the result is a sorted list bullet values returned as bulletsSorted. Since we can use all of Notes formula language, we could done @ReplaceSubstring or @Unique or whatever. The client app simply gets the data it needs in the form and format it needs them.
 
Imagine the following code. Can you see what it would do, and how you might be able to use it while building a web page result?
 
Body[Graphic *;%GraphicWidth]{AP}=widths,@Max($$widths.GraphicWidth)=maxWidth
 
 
Fancy trick 4: - Retrieving related data from other documents and databases
Anyone who has worked with Notes for a while will know that a view cannot do @DbLookups to get related information. If an item number is used but the item name, description, and price are in another database, you can do @DbLookups to retrieve them from inside a document but not from a view. In a typical REST API, you might do one request to get this document's information and then other calls to retrieve related information, but Exciton Boost allows you to use formulas, so you can retrieve related information in your first request. Rather than retrieving lots of data and dealing with it on the client, this also allows highly efficient view operations and caching to empower these lookups. As a simple example from our Curb Appeal database, we could use the following to display the average price of all houses in the current city.
 
City+", "+State{P}=key,@DbLookup("";"";"Listings\\by Location";$$key;2){P}=prices,@Text(@Sum($$prices)/@Elements($$prices);"C")=avg 
 
so, for a listing in Beachwood, OH, you would get
 
{
    "@unid""524C08BF648BE77A85257B8600798F5B",
    "avg""$460628.57"
}
 
Note that I used the previous Fancy Trick twice to save the key and then save the results of the lookup. Also note, when the values do not come from chunk properties but rather from formulas or items, the name is simply preceded by the two dollar signs. This process is very similar to saving variables in any language, but especially in cases where many data points are involved, it saves considerable time and bandwidth.
 
=> Request a free Exciton Boost eval in January, and if you later purchase a license. we'll add in four hours of development assistance, a $700 value. <=
 
If you have found these tricks interesting or intriguing, let me know and I can show more that relate to other parts of the REST API or remote procedure calls.
 

Copyright © 2021 Genii Software Ltd.

Tags:

Fri 22 Jan 2021, 06:51 PM
Inline JPEG image
 
 
I finally found time to put together two examples of "Fancy tricks" to do with Exciton Boost's REST API, and it is Friday afternoon, so I don't want to post it and have nobody notice. I'll post that Sunday night or Monday morning, but for now, let me leave you with a simple challenge. How would you use Domino Access Services or other HCL code to take the multiple rich text fields above and turn them into a sorted JSON array of strings for all the bullet values? It can obviously be done, but probably not as simply and elegantly as with Exciton. It took me about a minute to write out the request for this and far less than a second to get the following result. I'll show the request along with my post on Monday.
 
{
    "@unid""524C08BF648BE77A85257B8600798F5B",
    "bulletsSorted":
        "Attached parking",
        "Automatic garage door opener",
        "Basement",
        "Bedroom 2 is 15.00 x 14.00",
        "Bedroom 3 is 15.00 x 15.00",
        "Bedroom 4 is 16.00 x 13.00",
        "Central air",
        "Central vacuum",
        "Cul-de-sac location",
        "Dining Room is 15.00 x 14.00",
        "Dishwasher",
        "Eat-in kitchen",
        "Finished basement",
        "Fireplace in great room",
        "Forced air heat",
        "Formal Dining Room",
        "Foyer",
        "Full basement",
        "Garage spaces: 3",
        "Garbage disposal",
        "Great room is 23.00 x 22.00",
        "Great Room",
        "Heating: Gas",
        "Inground pool",
        "Irregular lot",
        "Kitchen is 23.00 x 18.00",
        "Library is 12.00 x 12.00",
        "Lot acreage is: 0.2650",
        "Master bedroom is 18.00 x 15.00",
        "Microwave oven",
        "Office",
        "Office is 16.00 x 15.00",
        "Other Room is 21.00 x 18.00",
        "Oven",
        "Patio",
        "Paved driveway",
        "Porch",
        "Property style: Colonial",
        "Public sewer",
        "Public water",
        "Rec Room is 30.00 x 12.00",
        "Refrigerator",
        "School district: Beachwood CSD",
        "Security system",
        "Smoke alarm",
        "Sun Room",
        "Sun Room is 17.00 x 16.00",
        "Warranty included",
        "Wooded lot",
        "Zoned heat"
    ]
}
 

Copyright © 2021 Genii Software Ltd.

Tags:

Tue 19 Jan 2021, 09:23 PM
Inline JPEG image
 
At first glance, it may seem like REST APIs are inherently simple. You retrieve the data from a record or collection of records, and then you do your stuff. If you need to create, update records, or delete records, you can do that as well. But when you are building a proper application, there are many other things to consider. Two key factors are:
 
1) How much data do you need from any given record, and which records do you need it from? If there is related data in another record, can you get it without pulling in loads of stuff you don't need? 
 
2) What format should the data be in? How should it be structured or grouped for maximum usability? While you can certainly write code in your application to reformat or restructure, that adds complexity and room for error on your client.
 
Using the Curb Appeal database I showed in Getting data from Domino with Exciton Boost (and companion post with code samples,Data from Domino: nuts and bolts of REST calls), let's look at a few of the "fancy tricks", also known as "useful features" of Exciton Boost 4.6.1 (released today). With Exciton Boost, you can specify exactly which information you need. In the code sample post above, I use the following call:
 
GET /CurbAppeal.nsf/api/boost/documents/524C08BF648BE77A85257B8600798F5B?metadata=false HTTP/1.1
X-Items-List: StreetAddress+", "+City+", "+State=Address,Bdrms,Full=Bathrooms,Half=Half-Baths,Photo[Graphic 1]
 
The resulting JSON is shown below.  Note that we used formula language for the address, but I've shown that before.
Inline JPEG image
 
Fancy trick 1: - Grouping stuff together
If you look up at the original screen capture, you can see that the bathrooms are grouped together. That is because people logically expect full bathrooms and half bathrooms to go together. So, how can we make our JSON, and thus our eventual JavaScript object, reflect this natural grouping? Pretty simple, really. I just change the X-Items-List header to the following (changes shown in Blue).
 
X-Items-List: StreetAddress+", "+City+", "+State=Address,Bdrms,Full=Bathrooms/Full,Half=Bathrooms/Half,Photo[Graphic 1]
 
If successive items are defined with that =group/alias syntax, the result will be grouped in an object named for the group. Now, we get this JSON, and can act on the whole Bathrooms object in our app.
Inline JPEG image
 
Fancy trick 2: - Using rich text as a data source rather than rendering
If you look up at JSON right above the Fancy trick 1 headline, you can see that it is HTML encapsulated in JSON. That is handy if you want to display it, but not if you want to use it as a data object. You could by parsing the HTML, but you shouldn't have to. A similar example is in the columns of bullet points you can see in the top graphic under the title Features. What if you want those bullet points as separate data strings rather than HTML? Exciton Boost makes it easy to define the data structure you want rather forcing you to parse and interpret.
 
We change the reference from Photo[Graphic 1] to Photo[Graphic *] to mean all graphics in the rich text field named Photo. Then, we add in a series of chunk properties related to a graphic, ;$GraphicOffset;$GraphicFormat;%GraphicHeightPX;%GraphicWidthPX. These will all be grouped together. Finally, we add the {A} which makes this an array of objects, one per graphic. If there were no graphics, it would be an empty array.
 
X-Items-List: StreetAddress+", "+City+", "+State=Address,Bdrms,Full=Bathrooms/Full,Half=Bathrooms/Half,Photo[Graphic *;$GraphicOffset;$GraphicFormat;%GraphicHeightPX;%GraphicWidthPX]{A}
 
Since there is only one graphic in this rich text field, the result looks like this:
 
Inline JPEG image
 
We can iterate through the array to get to the objects and their values, but if we don't want to do that, we could simply change the {A} to {O}, and we would get an object with named sub-objects:
 
X-Items-List: StreetAddress+", "+City+", "+State=Address,Bdrms,Full=Bathrooms/Full,Half=Bathrooms/Half,Photo[Graphic *;$GraphicOffset;$GraphicFormat;%GraphicHeightPX;%GraphicWidthPX]{O}
 
Inline JPEG image
 
Now, we could use JSON.parse to get the JavaScript object and refer to Photo.Graphic_1.GraphicHeightPX to get the value 251. As a note, capitalization follows the item definitions, so if you prefer camel case, just make it:
 
X-Items-List: StreetAddress+", "+City+", "+State=address,bdrms,Full=bathrooms/full,Half=bathrooms/half,photo[graphic *;$graphicOffset;$graphicFormat;%graphicHeightPX;%graphicWidthPX]{O}
 
Inline JPEG image
 
Finally, it is worth noting that if only a single property is specified, we don't need objects. If we wanted to get all the bullet items under that Features column. we could use:
 
X-Items-List: StreetAddress+", "+City+", "+State=address,bdrms,Full=bathrooms/full,Half=bathrooms/half,col1[BulletList *;$UnformattedText]{A}=bullets
 
Inline JPEG image
 
=> Request a free Exciton Boost eval in January, and if you later purchase a license. we'll add in four hours of development assistance, a $700 value. <=
 
I will have a couple more Fancy Tricks tomorrow.
 

Copyright © 2021 Genii Software Ltd.

Tags:

Mon 11 Jan 2021, 10:37 AM
Continuing on the development challenge outlined in REST plus RPC: Do the demo backward and started in REST plus RPC: the right data in the right format, and no more, this post describes how we can build up the JavaScript necessary to invoke a batch of remote procedure calls with the JSON-RPC format. There are multiple ways to send HTTP requests and get back data in JavaScript. I find it easy to understand fetch, so that is what I am using, but it should be easy to translate to jQuery or Ajax or whatever. In this case, the devil is not in the details, but in the broad picture.
 
Note: I am not a JavaScript expert. Some of you will know more than I do, or have different ways to accomplish the task. If you'd like, comment and tell me how I could do differently or better.
 
Remote procedure calls with Exciton Boost use JSON-RPC. I did a Quick primer on JSON-RPC a few months ago if you are want to understand it better, but for our purposes, simply understand that we POST a message with a JSON body in a particular format to call procedure calls, and if an id is specified, we get a return value. There may be an array of methods called sequentially. For example, if I want to get the database title for the ReportIt.nsf database in our proposed demo, I would POST the following:
 
{
  "jsonrpc": "2.0",
  "method": "db.getTitle",
  "params": null,
  "id": 1
}
 
and would get back the JSON payload
 
{
    "jsonrpc""2.0",
    "result""Report It!",
    "id"1
}
 
Now, let's look at a sample JavaScript fetch way of POSTing that. Since I want to show a sequence of calls, I'll also get the default form.
 
const getTitle =  { jsonrpc: "2.0",  
                    method: "db.getTitle",
                    params: null,
                    id: 1
                  };
 
const getForm =   { jsonrpc: "2.0",  
                    method: "db.getDefaultForm",
                    params: null,
                    id: 2
                  };
 
 
function doIt() {
 
var data = [getTitle,getForm];
var stat = 0;
 
fetch('/ReportIt.nsf/api/boost/rpc', {
  method: 'POST', 
  mode: 'same-origin',
  cache: 'no-cache',
  headers: {
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(data),
})
.then(response => {stat = response.status; return response.json();})
.then(data => {
if (stat == 200) { 
  document.getElementById("demo").innerHTML = '<h1>'+data[0].result+'</h1>Default form: '+data[1].result;
} else { 
  document.getElementById("demo").innerHTML = '<h1>Status: '+stat+'</h1>'+data[0].error.message;
}});
 
 
};
 
For the crude purposes of this demo, I put a button in HTML that sets the value of a div with id='demo'.
 
<button type="button" style="border-radius: 6px;" onClick="doIt();">Get the database title</button><br>
<p id="demo"></p>
 
When we put this all together on a demo Page element in our database, we get the following. We can build on this for our demo in the next post.
 
Inline GIF image
 
=> Request a free Exciton Boost eval in January, and if you later purchase a license. we'll add in four hours of development assistance, a $700 value. <=
 
 

Copyright © 2021 Genii Software Ltd.

Tags:

Wed 6 Jan 2021, 12:24 PM
Following up on the promise in yesterday's post, REST plus RPC: Do the demo backward, this post describes our first steps in building the nested tabbed table report demo.
 
My wife works in a preschool, and the first lesson for a successful day is: Rest first (well-rested and well-fed kids are happy kids). Most often, we find the same lesson is true when building apps which may require both the REST API and remote procedure calls: REST first (collect the necessary data, and only that data, in the most usable format).
 
Let's examine the data requirements by looking at a snippet of the table and marking up what data we'll need, some item values and other formulas.
 
Inline JPEG image
 
For those who have been following along, there are direct parallels  from my earlier post, Data from Domino: nuts and bolts of REST calls. That post is a good place to start when thinking about extracting data from any database using Exciton Boost's REST API. It is also a good post for showing many different ways to call the REST API from different languages. In this post, I'll stick to HTTP for clarity.
 
Task 1: Get list of accessible views
Returns a JSON array of objects, one for each view that is included, implicitly (all views) or explicitly (specified views), in the Exciton configuration db and marked as discoverable. Exciton is strict about data security, so views may be accessible but not discoverable. Views not included are inaccessible and will return a 404 error.
 
GET /ReportIt.nsf/api/boost/views HTTP/1.1
Accept: application/json
 
One of the accessible views is (AllOrdersByState) which has a UNID of 8F4CC3794598B864852586540074DE3F. We could access it using the UNID, but since using view names was added in Exciton Boost 4.6.0, we'll do that instead.
 
Task 2: Get view with columns & metadata
Returns a JSON array of objects, one for each document in the view. Each document includes a link to get the document from the documents collection as well as item values for each view column. Note that there are parameters to page through the view if desired.
 
GET /ReportIt.nsf/api/boost/views/(AllOrdersByState) HTTP/1.1
Accept: application/json
 
We get a JSON array of objects like that below. From this, we know we'll need State, City, and Qty, but won't need anything else.
 
Inline JPEG image
 
Task 3: Add selected items to view results 
Going back to the view, the request can include either additional or replacement items for the view columns using an item list via URL or HTTP header. This allows retrieval of any allowed items with the view results, including rendered rich text items.
 
GET /ReportIt.nsf/api/boost/views/(AllOrdersByState)?metadata=false HTTP/1.1
Accept: application/json
X-Items-List: State,City,LastName+", "+FirstName=Saleperson,Unit,Quantity,Color,DeliveryMethod,@Text(Price;"C,")=Price
 
And now we have the data we need in the format we need without the extra baggage. This call is the one REST call that is necessary, though it only returns 20 documents in the view at a time by default. That's a matter for another post as we process the data further
 
Inline JPEG image
 
 
=> Request a free Exciton Boost eval in January, and if you later purchase a license. we'll add in four hours of development assistance, a $700 value. <=
 
 
Continuing reading as I develop the solution. My follow up post, REST plus RPC: building the JavaScript for RPC calls, is now available.

Copyright © 2021 Genii Software Ltd.

Tags: