Paste Text from Clipboard feature

[BrownSky]

Hi, apologies if this shouldn't be under Bugs, I thought about putting it under New Features, but decided the latter is for not-yet-implemented stuff, whereas this has now been implemented, so belongs here.

This is a great feature, and is 99% of the way there to being usable.

It obviously does special handling for line breaks (Ascii codes 13 and 10, either separately or together in a certain order),
in order to know when to wrap to a new line, e.g.:














However it also places a question mark text element to represent a line break, but I think it should not do this,
that is to say,
it should wrap the line but NOT place a question mark text element,
instead it should leave whatever element was there already, as is.

Secondly Tab characters (Ascii code 9) also get translated to a question mark text element.
I think they should just be ignored, i.e. leave whatever element was there already, as is.

Forn the two situations:

- 1. Line breaks, Ascii 13 and 10, either on their own or as 13 followed by 10,
- 2. Tabs, Ascii 9,

The difference between how I would see them being handled under my proposal, is:

For the first case, the 'active insertion point' moves vertically down a row and back to the first column in which text entry occurred;

Whereas in the second case, the 'active insertion point' moves horizontally to the right by one column.

In both cases, whatever element was there to start with, would not be overwritten.

Thinking about this, I cannot see any difference between what I am asking for the Tab character above, and what I would also ask for Space characters,
i.e. I would like to map Space characters not to the Space text element, but to 'skip and move right',
i.e. leaving what is there already.
(I think we talked about this in the post 'Changing between 2 kinds of text elements'.)
But I can see other people might want different,
or even sometimes I might want to overwrite existing elements, rather than skip over them.
So maybe it could be handled by an option?
Even under the 'Editor' tab in the Config for a level
But I appreciate this would be a bit of work.

In fact, taking my suggestions above to the logical extreme, I suggest that the 'keep existing and skip rightwards' behaviour be extended from Ascii code 9 to ALL Ascii codes from 0 to 32 (Space char), except for 13 and 10.
This would enable the use of mucky bits of text from unusual sources [such as a RnD level opened in certain text editors, or a byte editor ]

And Ascii code 127, which I think is Backspace, could be used to 'reverse' the 'active insertion point one column to the left?

But [more] seriously, the most important thing is to get rid of the multitude of question mark text elements that flood the level when I use this feature, and also cause an increase in entropy, because there is no way of retrospectively distinguishing between question mark text elements that have been legitimately added [i.e. are derived from question marks in the source text] and those that simply indicate 'couldn't be mapped'.

I tried to do it myself, by setting the Replace feature to 100% and only replace question mark text elements, but this destabilizes RnD, first it replaces all instances of Empty Space in the palette with a wierd element [looks like normal wall but isn't], and then when I click on some buttons, eg Test, it causes RnD to instant-quit

John

[filbo]

Sounds like some refinements are definitely in order.

Your paste pipeline is apparently sending in CR (0x0D) and LF (0x0A), while Holger only expected one of those (probably LF). So, RnD should silently absorb one or both of those.

Your example shows some '?' in left column (well, left column of what I assume is the rectangular area you were targeting with your paste). But not every line starts with one, so I deduce that these are at the end of each line (we can't see the right-ends of your '#include' etc. lines). So, that makes sense: the input stream would have been like '#include <foo.h><CR><LF><CR><LF><CR><LF>#define blah...' -- in which CR generates a '?' at the end of each line (which also happens to be the beginning, on the two blank lines); and LF moves on to the beginning of the next line.

If you can train your paste-pipeline to omit CR characters, that would be fixed. And RnD should do the same: ignore CR. Except then there will be systems (Mac?) where each line-end comes with just a CR, no LF, so then things get complicated. I think a reasonably strong state machine implementation is: when either CR or LF is received, move to beginning of next line; also record which line-end char was received. Then if another same-line-end char is received, move to the next line, etc. But if the 'other' line-end char is received, clear the which-received flag and do not move.

Thus, the sequences 'A<CR>B', 'A<LF>B', 'A<CR><LF>B', 'A<LF><CR>B' all generate two 1-char lines; while 'A<CR><CR>B', 'A<CR><LF><CR>B', and various other combinations you can derive, generate 2 1-char lines with a blank line in between. Real-world inputs will always have whatever the local OS uses for line-ends, i.e. 'LF', 'CRLF', 'CR', or (VERY rarely) 'LFCR', at the end of each line, with no weird mixes. Doesn't matter; the state machine absorbs them all with equanimity.

And you're sending TAB characters. For that, primarily, I would suggest that you shouldn't be sending TAB characters! Deal with that at your source end.

But, of course, RnD should also do something sensible with those. Which I would suggest is either: ignore completely; or move one space forward (not overwriting a grid square, just skipping it).

Spaces, I think, should be handled as 'put a space of the currently-being-written character set into the grid square'. Otherwise you would have unpleasantly weird results from pasting 'this is a test' into the middle of some noisy field.

=====

I can see that you were testing pastes from C source code; and one might semi-reasonably(*) expect to see TAB chars in C source. But, only in the very weirdest of cave levels would I expect to see C source pasted into the gameboard! Probably real sources you might use will not have this issue in the first place.

(*) Except I am vehemently against TABs in source code. Makefiles are an unfortunate exception (syntactically required).

TABs in source files mean that the same source displays differently in every editing / viewing tool, differently in every user's editor. And they can leave behind invisible detritus (stuff like 'space space TAB space TAB TAB' -- you can't see the spaces, but they can have a real effect in some situations). Any project with >1 person editing code should shun TABs. I mean, it should be in the project rules: NO TABs, PERIOD, except in Makefiles or any other obscure example where they're required by the file's language syntax. The 'commit to source code management' pipeline (`git commit` or whatever) should have an active 'hook' that errors out if it smells TABs.

[filbo]

(oh, and of course it should be under bugs :)

[BrownSky]

I don't know how to quote from other users' posts or do attachments etc so apologies in advance for blunders.

Your paste pipeline is apparently sending in CR (0x0D) and LF (0x0A), while Holger only expected one of those (probably LF). So, RnD should silently absorb one or both of those.

I think Holger has implemented it handling CRLF, and the 2 characters are treated as one when pasting this way; so the question mark element is a response to CRLF, rather than to LF alone. I say this because the feature also handles Tabs, which are a single character, this way.

Your example shows some '?' in left column (well, left column of what I assume is the rectangular area you were targeting with your paste). But not every line starts with one, so I deduce that these are at the end of each line (we can't see the right-ends of your '#include' etc. lines). So, that makes sense: the input stream would have been like '#include <foo.h><CR><LF><CR><LF><CR><LF>#define blah...' -- in which CR generates a '?' at the end of each line (which also happens to be the beginning, on the two blank lines); and LF moves on to the beginning of the next line.

Yes I think I pasted from RnD, \src\editor.c
from the start, for about 100 lines.
i.e.

// ============================================================================
// Rocks'n'Diamonds - McDuffin Strikes Back!
// ----------------------------------------------------------------------------
// (c) 1995-2014 by Artsoft Entertainment
// Holger Schemel
// info@artsoft.org
// https://www.artsoft.org/
// ----------------------------------------------------------------------------
// editor.c
// ============================================================================

#include <math.h>

#include "libgame/libgame.h"

#include "editor.h"
#include "screens.h"
#include "tools.h"
#include "files.h"
#include "game.h"
#include "init.h"
#include "tape.h"


#define INFOTEXT_UNKNOWN_ELEMENT "unknown"


// ----------------------------------------------------------------------------
// screen and artwork graphic pixel position definitions
// ----------------------------------------------------------------------------

// values for the control window
#define ED_CTRL1_BUTTONS_HORIZ 4 // toolbox
#define ED_CTRL1_BUTTONS_VERT 4
#define ED_CTRL2_BUTTONS_HORIZ 3 // level
#define ED_CTRL2_BUTTONS_VERT 2
#define ED_CTRL3_BUTTONS_HORIZ 3 // CE and GE
#define ED_CTRL3_BUTTONS_VERT 1
#define ED_CTRL4_BUTTONS_HORIZ 2 // CE and GE
#define ED_CTRL4_BUTTONS_VERT 1
#define ED_CTRL5_BUTTONS_HORIZ 1 // properties
#define ED_CTRL5_BUTTONS_VERT 1
#define ED_CTRL6_BUTTONS_HORIZ 3 // properties
#define ED_CTRL6_BUTTONS_VERT 1
#define ED_CTRL7_BUTTONS_HORIZ 1 // palette
#define ED_CTRL7_BUTTONS_VERT 1

#define ED_NUM_CTRL1_BUTTONS (ED_CTRL1_BUTTONS_HORIZ * ED_CTRL1_BUTTONS_VERT)
#define ED_NUM_CTRL2_BUTTONS (ED_CTRL2_BUTTONS_HORIZ * ED_CTRL2_BUTTONS_VERT)
#define ED_NUM_CTRL3_BUTTONS (ED_CTRL3_BUTTONS_HORIZ * ED_CTRL3_BUTTONS_VERT)
#define ED_NUM_CTRL4_BUTTONS (ED_CTRL4_BUTTONS_HORIZ * ED_CTRL4_BUTTONS_VERT)
#define ED_NUM_CTRL5_BUTTONS (ED_CTRL5_BUTTONS_HORIZ * ED_CTRL5_BUTTONS_VERT)

and so on, but above is what corresponds to in the screenshot of the level I have attached, see below.

And you're sending TAB characters. For that, primarily, I would suggest that you shouldn't be sending TAB characters! Deal with that at your source end.

But, of course, RnD should also do something sensible with those. Which I would suggest is either: ignore completely; or move one space forward (not overwriting a grid square, just skipping it).

Yes, moving forward/to the right just 'fits' with the concept of a Tab.

The real-world situation for encountering Tabs is, if you copy from a spreadsheet; every cell has a Tab following it, followed [optionally] by another cell, and so on. Not sure if trailing Tabs are excluded by Windows Clipboard.

The Tabs are because spreadsheet cells contain variable-length data, unlike RnD element token values [numeric], but the same as RnD element token values [string], which have now been implemented as a means of data transfer, but which I would like to keep out of this post.

Even if Tabs were interpreted as I want, pasting from a spreadsheet into RnD would 'explode' the result, by putting an additional column between each non-tab character. Which might be what is wanted sometimes, but not usually.

I can deal with this by pasting into a text editor and removing all Tabs, for sure.

Spaces, I think, should be handled as 'put a space of the currently-being-written character set into the grid square'. Otherwise you would have unpleasantly weird results from pasting 'this is a test' into the middle of some noisy field.

This is where we most definitely disagree; I love the idea of 'merging' text into preexisting levels, as opposed to just a 'replace all', which is what a paste usually is.
My suggestion is to have the ability to do both, maybe with different commands (Ctrl+V vs Ctrl+Shift+V? Ctrl+Alt+V?)

TABs in source files mean that the same source displays differently in every editing / viewing tool, differently in every user's editor. And they can leave behind invisible detritus (stuff like 'space space TAB space TAB TAB' -- you can't see the spaces, but they can have a real effect in some situations). Any project with >1 person editing code should shun TABs. I mean, it should be in the project rules: NO TABs, PERIOD, except in Makefiles or any other obscure example where they're required by the file's language syntax. The 'commit to source code management' pipeline (`git commit` or whatever) should have an active 'hook' that errors out if it smells TABs.

[/quote]

I only use VBA, and in many situations the VBIDE replaces Tabs with runs of Spaces, typically 4. But no I wouldn't see any need to paste from the VBIDE into RnD. I see I made a mistake by using a source code file to test; I should have gone ahead with a spreadsheet example, which is much more real-world, because a spreadsheet can be used to do level operations that are difficult in RnD, e.g.

- insert rows between rows
- insert columns between columns
- autogenerate a level, 15 by 15, containing one each of elements 000 to 224
- drag and drop blocks of cells, instead of only being able to copy-paste them, as with the Grab Brush [apologies if it does have a cut-paste feature and I haven't learned it]
- delete n rows or no columns from anywhere in the level
- do scripted operations, eg a series of replaces


I will try to rustle up a spreadsheet example sometime.

John

[BrownSky]

A comment on the screenshot above: looking at all the occurrences of:

- the question mark text element
- the space text element

There isn't ONE case where any of these is adding value to what I want to do, or is making the level better, in my opinion. I get that's subjective, and others may value those space text elements - but surely not the question mark text elements? - but my feeling is:

The question mark text elements form an impenetrable wall along the left edge;
if Rockford was coming down the lefthand side - remember there are more columns out of sight on the left -
- and the question mark text elements were NOT there,
then to get inside the 'zone of text elements', he could go in at any of several lines.
Because they are there, he cannot get in anywhere:

[current, 'bad' or at least suboptimal:]


























[vs if it was 'good':]


























He COULD get someway into the block of text if he went right to the bottom of the block
[assuming the block didn't go down to the bottom of the level],
along to the right to the end of the block [assuming it didn't go to the last column,
up, [assuming the blocks have different numbers of characters, which is a good assumption for computer code, but maybe not for everything],
and then in from the right hand side.
- where he would encounter yet more question mark text elements:




















As for the space text elements,
if they were not there,
he could travel in and up,
eg between all those #INCLUDEs and the "" words to the right of them:

[good]
















[bad]
















perhaps to nab a necessary jewel,
perhaps pursued by a firefly or robot or two?
Much more fun.

No, big solid blocks of wall-like material are not much fun for me.

The more the feature promotes game-like behaviour, even when the source material is computer source code, the better!!! So say I.

John

[BrownSky]

The screenshot of 'going in from the right' looks like there is no way to get to the far left, due to all the randomly placed pipe which is facing the wrong way, but actually the very top two lines offer a way to get to the very far left cell, or they would IF the question mark text element was not there.

[BrownSky]

Attached shows where character text elements added unnecessarily [IMHO] into a level already containing randomly placed elements, have diminished the scope of the level for exploration and play.

[Marked with hand-drawn yellow Xes.]

I have not marked some text elements around the edge as the cut off view makes it impossible to tell if their presence or absence would make a meaningful difference or not.

John