This started as notes to try to solve a problem I’ve developed over time with making multiple websites/web-based projects. That problem is re-use of code. I keep making similar database structures, user-authentication systems, etc, and it really saps time away from developing whatever is unique about a particular project when I keep having to implement stuff I’ve already done. Even copying it over from an existing repository can be draining.
There’s a design pattern for this called Service Locator. The concept is to abstract the requirement of a service from its concrete definition. To be able to get something I know I need without having to know in code exactly where it is. This wouldn’t be a problem if everything was in one big project, but my code ain’t that simple. Importing pieces of projects into a new one and then using them requires that they aren’t necessarily in the same logical space all the time.
So let’s start with
What problems am I trying to solve?
- I don’t know where a model is. I’d like to just
import Name from require "models"but it could be defined anywhere. (Models are a convenience for accessing the database provided by Lapis. Some parts of my application are unique to it, and indeed inmodels, but there are parts they may be provided by any subset of sub-applications. Also, those parts may be imported into different locations, so they don’t even KNOW wheremodelsis!) - Sub-applications have no idea where they are. (On the main application, I can do things like the aforementioned importing from models, and
return render: "some_view"and they correct file will be fetched from the correct place. On a sub-application, because it can be located anywhere, I have to find a way for it to know where it is when it needs to.) - Migrations from updates to sub-applications won’t just be applied, and requires code duplication. This is somewhat a separate issue, but related enough I feel I should design a solution that works for it as well. (Related to this, migrations really should be based on UTC time stamps instead of just incrementing a number. I can’t solve this without doing that. I’ve known this from the start but been lazy.)
- I am re-implementing the exact same utility functions/library in all projects. This really should be its own project, and as easy to locate and access as anything else.
- Whatever I decide to do needs to have a legacy import option to be able to be dropped into existing projects without breaking them. (For example, I’ll be switching how migrations work, and if a project was already using one of these projects, it would try to run migrations the duplicate what’s already been done. Or, there’s the fact that I’ve changed how certain authentication systems work over time.)
- Shared state / Avoiding duplicate hits to the database. I know I’m going to want to run certain actions/filters on every request, or a subset of all requests (for example, checking if a user is an administrator is a common occurrence). Especially with user interaction, there are cases where I end up pulling data more than once, and can accidentally break something if I’m not very careful. It’d be nice to have a system that makes it easier to work with such things while guaranteeing that things won’t be broken by accessing them multiple times.
- How does the locator know where something is?
- Even once this is in-place, there are certain common operations I will be doing all the time, and it’d be nice to make a script to make those operations easier. For example, updating subtree’d projects (both in terms of updating from remote, and pushing to the remote). I’d like to be able to just run
import users, githookfrom the command-line instead of messing around with all the commands I know I’ll need. - I need to be able to expand and extend it like any other project, without breaking things, and I will forget to extend it on certain projects and then try to use newer features, so I need to make this easy to detect and fix.
What sacrifices will I have to make?
First off, I know I’m going to modify the default models file that comes with Lapis. There’s no way around it, because I want to just import from models with impunity, not caring where the model is located. But I don’t want everything to be fixed in that file, because then I’ll have ridiculousness like importing a random utility function from models, which is for, you know, models.
Second, whatever other file I define for this has to be located in the same location in every project from now on, because without that, nothing can access the locator to find other things!
While I do work on Windows, I tend to work more often on Linux, so I’m fine with making scripts and such that only run on Linux. (For example, I am most familiar with Lua, so making Lua scripts without an extension, a #!/usr/bin/lua (or whatever it is) at the top, and chmod‘ing them is sufficient for my scripting needs. Then again, for the simpler stuff, I might as well just use .sh files with #!/bin/bash…
What do I want usage to look like?
For sub-applications, the most important thing is for them to know WHERE they are before they can do anything:
-- v1
import pwd from require "locator" -- almost named this location, but pwd is easier to type and seems more appropriate
path = pwd(...)
import user_view from require "#{path}/views" -- now this is possible
-- v2
locator = require "locator"
import user_view from locator(..., "views")
-- v3
locate = require("locator")(...) -- returns a function that can be used to build a locally-scoped require-like function
import user_view from locate "views" -- I like this version most because ease of use
Where the hell are my models? In order to find out, the locator has to have some concept of storing where to look..and somehow also has to know where to look for anything at all times in order to work no matter what order things are called in. (I can’t do something like locator.registerModelPath(..., "models").) One solution is to have two always-in-the-same-place files, the locator itself, and a config file for paths that it will require (as well as anything else I may need). My only concern with this idea is with updating the locator itself.
Related to that, I’ve been doing a bunch of reading about how Lua handles modules and packages because I was confued about how Lua knows to translate require('name') to ./name/init.lua sometimes, but not other times. Seems to be it’s a feature of the Love engine that I didn’t realize wasn’t part of standard Lua (standard Lua only looks in special module installation paths based on package.path, my guess is Love’s implementation defines a game’s source code directory as one of these paths, making that function there, and nowhere else).
I think as a solution to being able to later expand or extend the locator in the future, the locator should be in a directory as a init.lua file (when compiled, I will be writing it in MoonScript), and you will need to manually add a file with the same name as that folder with return require((...) .. '.init') in it. This doesn’t solve the config file though. I can’t add it to a gitignore, and can’t leave it hanging inside a subtree. Config will have to be just another file… OR could just be incorporated into Lapis’s config system if I figure out exactly how that works. Lapis’s config system is available during server execution, but that happens separately from migrations.
Now, that doesn’t solve things for models, unless I also include a small shim for it to require something inside of the locator project.
I lost my train of thought so here’s a shortened less-train-of-thought’d list of things I need to do
import Name from require "models"- Sub-applications have no idea where they are.
- Migrations from updates to sub-applications.
- Easy access to the utility stuff I always have.
- Needs to have a legacy import option to be able to be dropped into existing projects without breaking them. (For example, migrations and authentication.)
- before filter needs to use something from the locator to extend the filter to sub-applications.
- Shared state / Avoiding duplicate hits to the database. Especially w users sub-application storing user in
@useron requests. - How does the locator know where something is?
- Commands to update subtrees / config would be nice.
- Ability to update as a subtree itself.
-- so now I'm gonna write some more "how I'd like to use it" code
locate = require("locator")(...) -- returns a function that can be used to build a locally-scoped require-like function
-- maybe the order of these should be attempting to use local scope first! (if locate is a fn grabbed from factory)
import user_view from locate "views" -- will try to use main views, then the views in each sub-application as specified by the config (haven't decided on config format yet)
import Model from locate "models" -- will try the main models table first, then in each sub-application (basically, one simple method for everything solves several problems at once)
-- NOTE the locator should handle searching for a folder with an init.lua inside of it and returning that!
-- locate "migrations" -- would just return the normal migrations table.. I want something a little better
locator = require "locator"
print locator.migrations -- this is a table of all migrations (as specified by config format (which will contain legacy-supporting options))
import some_func from locate "utility.string" -- this is how I would access the utility functions
-- legacy option for migrations would be specifying what key to START migrating from on specific sub-applications' migrations (so older keys are ignored)
-- NOTE to do this, I will be sorting migration tables
-- in order to support the before_filter ... I'm not sure yet
-- for the shared state / modifying a request... I think I leave this up to how sub-applications handle things, this was more of a note to myself about how I want to rewrite users
-- as for how the locator knows where something is, it tries a local scope first if specified, then it tries global, then it tries each registered sub-application (specified by unknown config format)
-- commands to update subtrees / config will be added later if they are added
-- as for it updating as a subtree, I need to re-read this whole document and start writing it
And that’s all for now.
This story doesn’t have an ending just yet, but I have a pretty solid foundation to decide how to proceed from.