How to delete files uploaded via Smart action?

Hello there,

I have a Smart action where I upload multiple files to S3. It works fine.
I’d like t to give the user the possibility to delete such files. How can I do that?
At the moment I created a smart field where the user can see the files, but not sure how to enable some action where the user can trigger a delete action.

Thanks

Depending on your use-case, you could remove the smart action, and deal everything from the smartfield by overriding the set and get handlers.

You can use an existing field of your model to store the path of the file in S3.

Handle the upload on set (the file will be provided as a datauri like with the smart action)
And return the full URL in get.

This works if using the adapted file widgets.
Would that suite your needs?

@romaing I tried to move everything to the smart field and it apparently works ok for the first part. I can see the list of assets (fetched from an S3 bucket):

But when I upload a new file or try to remove one from the list the set hook is never called.

Here’s how I defined the smart field:

{
      field: 'assets',
      type: ['File'],
      get: async (res) => {
        const objects = await S3.list({
          bucketName: process.env.ASSETS_BUCKETNAME,
          folder: `assets/${res.id}`,
        })

        const filenames =
          objects.Contents?.map(
            (object) =>
              `https://cdn.name/${
                res.id
              }/${object.Key.substr(object.Key.lastIndexOf('/') + 1)}`
          ) ?? []

        return filenames
      },
      set: (opts) => {
        // This function is never called
        console.log('SET HOOK', opts)

        return { sucess: 'OK' }
      },
    }

That is not normal.
Did you save the model?

I’ll try to find example code

Do you mean from the UI? If so, yes, I tried to upload or delete assets and when I press save everything works fine, but the set hook is never called, therefore I can’t deal with file uploads or deletion from the smart field.

Here is some sample code of a util to create smartfield that store files in S3 (and delete old files).
You need a string field on the model in database to store the path (as it’s randomly generated)

Note: Filenames will work only if you are using a V8 liana

Note 2: This might be buggy, I just pulled this code from an old repo :slight_smile:

const { S3Client, PutObjectCommand, DeleteObjectCommand } = require('@aws-sdk/client-s3');
const parseDataUri = require('parse-data-uri')
const { v4 } = require('uuid');
const models = require('../models');

/**
 * This function takes as parameter the name of a field and its collection
 * and creates a SmartField which will allow uploading files to S3.
 * 
 * @example
 * The following example will create a smartField named "bookFile" which
 * allows uploading files to S3.
 * The Amazon S3 key to the file will be stored in the "book" field.
 * 
 * collection('book', {
 *   fields: [
 *     smartFieldAmazonS3({
 *       field: 'cover',
 *       collection: 'book',
 *       region: 'us-east-2',
 *       accessKeyId: '...',
 *       secretAccessKey: '...',
 *       bucketName: 'mylibrary-forest-uploads'
 *     })
 *   ],
 * });
 */
function smartFieldAmazonS3(options) {
  const { sourceFieldName, collectionName, region, accessKeyId, secretAccessKey, bucketName } = options;
  const client = new S3Client({ region, credentials: { accessKeyId, secretAccessKey } });

  return {
    field: `${sourceFieldName}File`, // Create a new field with "File" appended to contain the file
    type: 'String',
    get: (record) => {
      return record[sourceFieldName]
        ? `https://${bucketName}.s3.${region}.amazonaws.com/${record[sourceFieldName]}`
        : null;
    },
    set: async (record, dataUri) => {
      // Delete previous file if needed.
      const oldRecord = await models[collectionName].findOne({
        attributes: [sourceFieldName], where: { id: record.id }
      });

      if (oldRecord[sourceFieldName]) {
        await client.send(new DeleteObjectCommand({ Bucket: bucketName, Key: oldRecord[sourceFieldName] }));
        record[sourceFieldName] = null;
      }

      // Upload new file
      if (dataUri) {
        const file = parseDataUriWithName(dataUri);
        const key = `${v4()}/${file.name}`;

        await client.send(new PutObjectCommand({
          ACL: 'public-read',
          Body: file.data,
          Bucket: bucketName,
          ContentType: file.mimeType,
          Key: key,
        }));

        record[sourceFieldName] = key;
      }

      return record;
    }
  }
}

/** 
 * Extract filename from data-uri.
 * Returns null if no filename was provided.
 */
function getPropertyFromDataUri(propertyName, dataUri) {
  const header = dataUri.substring(0, dataUri.indexOf(','));
  const part = header.split(';').find(p => p.startsWith(`${propertyName}=`));

  return part ? decodeURIComponent(part.substring(propertyName.length + 1)) : null;
};

function parseDataUriWithName(dataUri) {
  const file = parseDataUri(dataUri);
  file.name = getPropertyFromDataUri('name', dataUri) || 'unnamed';
  return file;
};

module.exports = { smartFieldAmazonS3 };

@romaing this is similar to what I’m doing and want to achieve. The problem is that the set hook function is never called for me so I can’t react to file uploads/deletion.
This is the version I’m using "forest-express-sequelize": "^7.11.1" Do I need to have some specific version for it to work?

Let me try to upgrade the version and see if at least I can get the set hook to work.

We never had bug reports about the set() hook not being called.
If the issue persist, I’ll downgrade to your version and test this locally

@romaing could you please downgrade your version locally and test it to check if you can reproduce it? I just started the process of upgrading to v8, but the amount of breaking changes introduced in v8 totally unrelated to what I’m trying to do makes it unfeasible to upgrade now. I’d need to refactor a lot of the codebase. I’d appreciate if you could check it.

FYI I tried to upgrade to at least the last minor version of v7 (7.12.2) and check if that would solve the problem, but it still persists, the set hook is never called.

Can you try changing the type of your smartfield to ['String']?
It should be with quotes ['String'] (not [String])

I tested locally, and everything is working on my end

I tried, nothing changes.
Here is the field declaration:

    {
      field: 'assets',
      type: ['String'],
      get: async (record) => {
        const objects = await S3.list({
          bucketName: process.env.ASSETS_BUCKETNAME,
          folder: `assets/${record.id}`,
        })

        const filenames =
          objects.Contents?.map(
            (object) =>
              `https://p.cdn/assets/${
                record.id
              }/${object.Key.substr(object.Key.lastIndexOf('/') + 1)}`
          ) ?? []

        return filenames
      },
      set: async () => {
        console.log('THIS HOOK IS NEVER CALLED')
      },
    },

The field itself works. I can see the files being added/removed from the UI, but when I press the save button the set hook for the smart field is never called.
I thought it might had something to do with the fact the this smart field doesn’t exist in the sequelize model. Both getters and setters are virtual. I even tried to add this field as a virtual field to the sequelize model thinking that maybe this could be the issue, but the problem persists.

I’ve been checking the source code.

Do you happen to either:

  • have a real sequelize column on the same collection called assets?
  • have completely overriden the PUT route for that collection?

If not I see no other way than to ask you adding breakpoints / console.log() on the file I’m linking in the forest-express module, as I’m not reproducing your issue, and the last commit that did something else than updating linting rules in this file was 4 years ago.

The call to .set is here: forest-express/resource.js at master · ForestAdmin/forest-express · GitHub

@romaing yes I override the put route because I’m calling an external API when the record is updated. Would that be a problem to not trigger the set hook?

The default PUT route is the one calling the SmartField injector, which in turn calls the .set hook.

IMHO you have two options:

  • Use sequelize hooks instead of overriding the route altogether to propagate the changes to your model
  • Override the route only to propagate the change, but call next() once that is done so that the default handler is called (this solution may prove more problematic for error handling depending on what you do in the route)

Can you show me the code of your overriden route?